Assessing Evidence Inconsistency in Mixed Treatment Comparisons
Abstract
Randomized comparisons among several treatments give rise to an incomplete-blocks structure known as mixed treatment comparisons (MTCs). To analyze such data structures, it is crucial to assess whether the disparate evidence sources provide consistent information about the treatment contrasts. In this article we propose a general method for assessing evidence inconsistency in the framework of Bayesian hierarchical models. We begin with the distinction between basic parameters, which have prior distributions, and functional parameters, which are defined in terms of basic parameters. Based on a graphical analysis of MTC structures, evidence inconsistency is defined as a relation between a functional parameter and at least two basic parameters, supported by at least three evidence sources. The inconsistency degrees of freedom (ICDF) is the number of such inconsistencies. We represent evidence consistency as a set of linear relations between effect parameters on the log odds ratio scale, then relax these relations to allow for inconsistency by adding to the model random inconsistency factors (ICFs). The number of ICFs is determined by the ICDF. The overall consistency between evidence sources can be assessed by comparing models with and without ICFs, whereas their posterior distribution reflects the extent of inconsistency in particular evidence cycles. The methods are elucidated using two published datasets, implemented with standard Markov chain Monte Carlo software.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The transitivity assumption was evaluated by comparing key study characteristics and assessing local and global consistency.
“The transitivity assumption was evaluated by comparing key study characteristics and assessing local consistency (net splitting of direct evidence from head-to-head studies vs indirect evidence inferred through common comparators) and global consistency (design-by-treatment interaction test).”
Cites
Gelman, A., Carlin, J. B., Stern, H. S., Dunson, D. B., Vehtari, A., & Rubin, D. B. (2013) · doi:10.1201/b16018 · Cited by 27,276 · see citing works
Network meta‐analysis for indirect treatment comparisons
Lumley, T. (2002) · Statistics in Medicine · doi:10.1002/sim.1201 · Cited by 1,206 · see citing works
Meta‐Analysis Of Controlled Clinical Trials
Whitehead, A. (2002) · doi:10.1002/0470854200 · Cited by 827 · see citing works
Bayesian approaches to random-effects meta-analysis: a comparative study
Smith, T. C., Spiegelhalter, D. J., & Thomas, A. (1995) · Statistics in Medicine · doi:10.1002/sim.4780142408 · Cited by 568 · see citing works
Warn, D. E., Thompson, S. G., & Spiegelhalter, D. J. (2002) · Statistics in Medicine · doi:10.1002/sim.1189 · Cited by 277 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Hutton, B., Salanti, G., Caldwell, D. M., Chaimani, A., Schmid, C. H., Cameron, C., Ioannidis, J. P. A., Straus, S., Thorlund, K., Jansen, J. P., et al. (2015) · Annals of Internal Medicine · doi:10.7326/m14-2385 · Cited by 8,269 · see citing works
Salanti, G., Ades, A. E., & Ioannidis, J. P. A. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.03.016 · Cited by 3,766 · see citing works
Self-citationChecking consistency in mixed treatment comparison meta-analysis
Dias, S., Welton, N. J., Caldwell, D. M., & Ades, A. E. (2010) · Statistics in Medicine · doi:10.1002/sim.3767 · Cited by 2,521 · see citing works
Self-citationGraphical tools for network meta-analysis in STATA
Chaimani, A., Higgins, J. P. T., Mavridis, D., Spyridonos, P., & Salanti, G. (2013) · PLoS ONE · doi:10.1371/journal.pone.0076654 · Cited by 2,473 · see citing works
Consistency and inconsistency in network meta-analysis: concepts and models for multi-arm studies
Higgins, J. P. T., Jackson, D., Barrett, J. K., Lu, G., Ades, A. E., & White, I. R. (2012) · Research Synthesis Methods · doi:10.1002/jrsm.1044 · Cited by 2,098 · see citing works
Self-citationCINeMA: an approach for assessing confidence in the results of a network meta-analysis
Nikolakopoulou, A., Higgins, J. P. T., Papakonstantinou, T., Chaimani, A., Del Giovane, C., Egger, M., & Salanti, G. (2020) · PLOS Medicine · doi:10.1371/journal.pmed.1003082 · Cited by 1,935 · see citing works
Salanti, G. (2012) · Research Synthesis Methods · doi:10.1002/jrsm.1037 · Cited by 1,729 · see citing works
Evaluating the quality of evidence from a network meta-analysis
Salanti, G., Del Giovane, C., Chaimani, A., Caldwell, D. M., & Higgins, J. P. T. (2014) · PLoS ONE · doi:10.1371/journal.pone.0099682 · Cited by 1,402 · see citing works
Pharmacological interventions for smoking cessation: an overview and network meta-analysis
Cahill, K., Stevens, S., Perera, R., & Lancaster, T. (2013) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd009329.pub2 · Cited by 1,243 · see citing works
Dias, S., Sutton, A. J., Ades, A. E., & Welton, N. J. (2013) · Medical Decision Making · doi:10.1177/0272989x12458724 · Cited by 1,207 · see citing works
Self-citationWhite, I. R., Barrett, J. K., Jackson, D., & Higgins, J. P. T. (2012) · Research Synthesis Methods · doi:10.1002/jrsm.1045 · Cited by 1,114 · see citing works
Evaluation of networks of randomized trials
Salanti, G., Higgins, J. P., Ades, A., & Ioannidis, J. P. (2008) · Statistical Methods in Medical Research · doi:10.1177/0962280207080643 · Cited by 1,078 · see citing works
Self-citationConceptual and technical challenges in network meta-analysis
Cipriani, A., Higgins, J. P. T., Geddes, J. R., & Salanti, G. (2013) · Annals of Internal Medicine · doi:10.7326/0003-4819-159-2-201307160-00008 · Cited by 1,072 · see citing works
Ioannidis, J. P. A. (2009) · Canadian Medical Association Journal · doi:10.1503/cmaj.081086 · Cited by 716 · see citing works
Fu, R., Gartlehner, G., Grant, M., Shamliyan, T., Sedrakyan, A., Wilt, T. J., Griffith, L., Oremus, M., Raina, P., Ismaila, A., et al. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.08.010 · Cited by 730 · see citing works
Comparison of Weight Loss Among Named Diet Programs in Overweight and Obese Adults: A Meta-analysis
Johnston, B. C., Kanters, S., Bandayrel, K., Wu, P., Naji, F., Siemieniuk, R. A., Ball, G. D. C., Busse, J. W., Thorlund, K., Guyatt, G., et al. (2014) · JAMA · doi:10.1001/jama.2014.10397 · Cited by 704 · see citing works
Network meta-analysis: application and practice using Stata
Shim, S., Yoon, B.-H., Shin, I.-S., & Bae, J.-M. (2017) · Epidemiology and Health · doi:10.4178/epih.e2017047 · Cited by 656 · see citing works
Jansen, J. P., & Naci, H. (2013) · BMC Medicine · doi:10.1186/1741-7015-11-159 · Cited by 656 · see citing works
Evaluation of inconsistency in networks of interventions
Veroniki, A. A., Vasiliadis, H. S., Higgins, J. P., & Salanti, G. (2013) · International Journal of Epidemiology · doi:10.1093/ije/dys222 · Cited by 576 · see citing works
Multivariate meta‐analysis: Potential and promise
Jackson, D., Riley, R., & White, I. R. (2011) · Statistics in Medicine · doi:10.1002/sim.4172 · Cited by 530 · see citing works
Network meta-analysis: application and practice using R software
Shim, S. R., Kim, S.-J., Lee, J., & Rücker, G. (2019) · Epidemiology and Health · doi:10.4178/epih.e2019013 · Cited by 483 · see citing works
A graphical tool for locating inconsistency in network meta-analyses
Krahn, U., Binder, H., & König, J. (2013) · BMC Medical Research Methodology · doi:10.1186/1471-2288-13-35 · Cited by 458 · see citing works
GetReal in network meta-analysis: a review of the methodology
Efthimiou, O., Debray, T. P. A., van Valkenhoef, G., Trelle, S., Panayidou, K., Moons, K. G. M., Reitsma, J. B., Shang, A., Salanti, G., & on behalf of GetReal Methods Review Group (2016) · Research Synthesis Methods · doi:10.1002/jrsm.1195 · Cited by 401 · see citing works
Psychological interventions to foster resilience in healthcare professionals
Kunzler, A. M., Helmreich, I., Chmitorz, A., König, J., Binder, H., Wessa, M., & Lieb, K. (2020) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012527.pub2 · Cited by 390 · see citing works
Naci, H., Salcher-Konrad, M., Dias, S., Blum, M. R., Sahoo, S. A., Nunan, D., & Ioannidis, J. P. A. (2019) · British Journal of Sports Medicine · doi:10.1136/bjsports-2018-099921 · Cited by 326 · see citing works
Zhou, X., Hetrick, S. E., Cuijpers, P., Qin, B., Barth, J., Whittington, C. J., Cohen, D., Del Giovane, C., Liu, Y., Michael, K. D., et al. (2015) · World Psychiatry · doi:10.1002/wps.20217 · Cited by 322 · see citing works
Jansen, J. P., Trikalinos, T., Cappelleri, J. C., Daw, J., Andes, S., Eldessouki, R., & Salanti, G. (2014) · Value in Health · doi:10.1016/j.jval.2014.01.004 · Cited by 319 · see citing works
Psychological interventions for resilience enhancement in adults
Helmreich, I., Kunzler, A., Chmitorz, A., König, J., Binder, H., Wessa, M., & Lieb, K. (2017) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012527 · Cited by 223 · see citing works
Uncertainty in Treatment Rankings: Reanalysis of Network Meta-analyses of Randomized Trials
Trinquart, L., Attiche, N., Bafeta, A., Porcher, R., & Ravaud, P. (2016) · Annals of Internal Medicine · doi:10.7326/m15-2521 · Cited by 202 · see citing works
A primer on network meta-analysis with emphasis on mental health
Mavridis, D., Giannatsi, M., Cipriani, A., & Salanti, G. (2015) · Evidence Based Mental Health · doi:10.1136/eb-2015-102088 · Cited by 198 · see citing works
Network meta‐analysis of multicomponent interventions
Rücker, G., Petropoulou, M., & Schwarzer, G. (2020) · Biometrical Journal · doi:10.1002/bimj.201800167 · Cited by 195 · see citing works
Escitalopram versus other antidepressive agents for depression
Cipriani, A., Santilli, C., Furukawa, T. A., Signoretti, A., Nakagawa, A., McGuire, H., Churchill, R., & Barbui, C. (2009) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd006532.pub2 · Cited by 190 · see citing works
Network meta-analysis: a norm for comparative effectiveness?
Higgins, J. P. T., & Welton, N. J. (2015) · The Lancet · doi:10.1016/s0140-6736(15)61478-7 · Cited by 133 · see citing works
Very early versus delayed mobilisation after stroke
Bernhardt, J., Thuy, M. N., Collier, J. M., & Legg, L. A. (2009) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd006187.pub2 · Cited by 134 · see citing works
Verde, P. E., & Ohmann, C. (2015) · Research Synthesis Methods · doi:10.1002/jrsm.1122 · Cited by 123 · see citing works
Adjustment for reporting bias in network meta-analysis of antidepressant trials
Trinquart, L., Chatellier, G., & Ravaud, P. (2012) · BMC Medical Research Methodology · doi:10.1186/1471-2288-12-150 · Cited by 122 · see citing works
Evaluating novel agent effects in multiple-treatments meta-regression
Salanti, G., Dias, S., Welton, N. J., Ades, A. E., Golfinopoulos, V., Kyrgiou, M., Mauri, D., & Ioannidis, J. P. A. (2010) · Statistics in Medicine · doi:10.1002/sim.4001 · Cited by 111 · see citing works
Self-citationGao, Y., Ge, L., Liu, M., Niu, M., Chen, Y., Sun, Y., Chen, J., Yao, L., Wang, Q., Li, Z., et al. (2022) · Sleep Medicine Reviews · doi:10.1016/j.smrv.2022.101648 · Cited by 89 · see citing works
Supervised Exercise Therapy and Revascularization for Intermittent Claudication
Saratzis, A., Paraskevopoulos, I., Patel, S., Donati, T., Biasi, L., Diamantopoulos, A., Zayed, H., & Katsanos, K. (2019) · JACC: Cardiovascular Interventions · doi:10.1016/j.jcin.2019.02.018 · Cited by 60 · see citing works
Yao, L., Sadeghirad, B., Li, M., Li, J., Wang, Q., Crandon, H. N., Martin, G., Morgan, R., Florez, I. D., Hunskaar, B. S., et al. (2023) · BMJ · doi:10.1136/bmj-2023-076226 · Cited by 57 · see citing works
Pescatello, L. S., Wu, Y., Gao, S., Livingston, J., Sheppard, B. B., & Chen, M.-H. (2021) · BMJ Open Sport & Exercise Medicine · doi:10.1136/bmjsem-2020-000895 · Cited by 50 · see citing works
Application of network meta-analysis in the field of physical activity and health promotion
Su, X., McDonough, D. J., Chu, H., Quan, M., & Gao, Z. (2020) · Journal of Sport and Health Science · doi:10.1016/j.jshs.2020.07.011 · Cited by 38 · see citing works
Zhang, T., Ye, X., Zhu, T., XIAO, X., Liu, Y., Wei, X., Liu, Y., Wu, C., Guan, R., Li, X., et al. (2015) · Medicine · doi:10.1097/md.0000000000001732 · Cited by 28 · see citing works
Veroniki, A. A., Tsokani, S., White, I. R., Schwarzer, G., Rücker, G., Mavridis, D., Higgins, J. P. T., & Salanti, G. (2021) · BMC Medical Research Methodology · doi:10.1186/s12874-021-01401-y · Cited by 18 · see citing works
Zhang, Y., Zhou, X., James, A. C., Qin, B., Whittington, C. J., Cuijpers, P., Del Giovane, C., Liu, Y., Cohen, D., Weisz, J. R., et al. (2015) · BMJ Open · doi:10.1136/bmjopen-2015-008572 · Cited by 10 · see citing works
Du, H., Bo, L., Lai, X., Zhu, H., & Huo, X. (2023) · Frontiers in Aging Neuroscience · doi:10.3389/fnagi.2023.1095996 · Cited by 10 · see citing works
Zhu, Z., Zhu, X., Chu, Y., Zhang, B., & Chen, Y. (2025) · EClinicalMedicine · doi:10.1016/j.eclinm.2025.103479 · Cited by 4 · see citing works
“28,29 Subgroup and sensitivity analyses were performed for the primary outcome to explore differences in RTI risk between adults aged 65 years and those ≥ 65 years, as well as changes in effect sizes after excluding high risk-of-bias studies, those with a “no treatment” control group, or…”
Li, Y., Liu, X., Guo, Z., Lai, L., Bryant, R. A., Zhang, T., Song, H., Mi, T., & Ren, Z. (2025) · Stress and Health · doi:10.1002/smi.70124
- Authors
- Lu, G., & Ades, A. E.
- Affiliations
- Royal Statistical Society (GB)
- ORCID
- Ades, A. E
- Venue
- Journal of the American Statistical Association
- Publisher
- Informa UK Limited
- Published
- 2006-06-01
- Volume
- 101
- Issue
- 474
- Pages
- 447-459
- ISSN
- 0162-1459 (ISSN-L)
- 1537-274X
- OpenAlex
- W1971895240
- Cited by
- 756
- References
- 5
- Citations by year
- 2026: 18 · 2025: 30 · 2024: 34 · 2023: 47 · 2022: 48 · 2021: 55 · 2020: 54 · 2019: 40 · 2018: 59 · 2017: 58 · 2016: 43 · 2015: 56 · 2014: 49 · 2013: 48 · 2012: 44
- FWCI
- 8.68
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Journal of the American Statistical Association (publishedVersion)
- Explore Bristol Research (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Keywords
- network meta-analysis, Bayesian hierarchical model, incomplete block designs, log odds ratio, model comparison
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
