The DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index.
Methodological problems in the use of indirect comparisons for evaluating healthcare interventions: survey of published systematic reviews
Open accessAbstract
OBJECTIVE: To investigate basic assumptions and other methodological problems in the application of indirect comparison in systematic reviews of competing healthcare interventions. DESIGN: Survey of published systematic reviews. Inclusion criteria Systematic reviews published between 2000 and 2007 in which an indirect approach had been explicitly used. DATA EXTRACTION: Identified reviews were assessed for comprehensiveness of the literature search, method for indirect comparison, and whether assumptions about similarity and consistency were explicitly mentioned. RESULTS: The survey included 88 review reports. In 13 reviews, indirect comparison was informal. Results from different trials were naively compared without using a common control in six reviews. Adjusted indirect comparison was usually done using classic frequentist methods (n=49) or more complex methods (n=18). The key assumption of trial similarity was explicitly mentioned in only 40 of the 88 reviews. The consistency assumption was not explicit in most cases where direct and indirect evidence were compared or combined (18/30). Evidence from head to head comparison trials was not systematically searched for or not included in nine cases. CONCLUSIONS: Identified methodological problems were an unclear understanding of underlying assumptions, inappropriate search and selection of relevant trials, use of inappropriate or flawed methods, lack of objective and validated methods to assess or improve trial similarity, and inadequate comparison or inappropriate combination of direct and indirect evidence. Adequate understanding of basic assumptions underlying indirect and mixed treatment comparison is crucial to resolve these methodological problems. APPENDIX 1: PubMed search strategy. APPENDIX 2: Characteristics of identified reports. APPENDIX 3: Identified studies. References of included studies.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Meta-analysis in clinical trials
DerSimonian, R., & Laird, N. (1986) · Controlled Clinical Trials · doi:10.1016/0197-2456(86)90046-2 · Cited by 39,552
The combination of estimates from different experiments
Cochran, W. G. (1954) · Biometrics · doi:10.2307/3001666 · Cited by 5,380
Interaction revisited: the difference between two estimates.
Altman, D. G., & Bland, J. M. (2003) · BMJ · doi:10.1136/bmj.326.7382.219 · Cited by 3,203
Self-citationClinical Trials: A Practical Approach.
Hill, I. D., & Pocock, S. J. (1984) · Biometrics · doi:10.2307/2531186 · Cited by 2,930
Combination of direct and indirect evidence in mixed treatment comparisons
Lu, G., & Ades, A. E. (2004) · Statistics in Medicine · doi:10.1002/sim.1875 · Cited by 2,166
Bucher, H. C., Guyatt, G. H., Griffith, L. E., & Walter, S. D. (1997) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(97)00049-8 · Cited by 2,052
Simultaneous comparison of multiple treatments: Combining direct and indirect evidence.
Caldwell, D. M., Ades, A. E., & Higgins, J. P. T. (2005) · BMJ · doi:10.1136/bmj.331.7521.897 · Cited by 1,600 · cited 1×
“To improve statistical power evidence generated by indirect comparison can be combined with evidence from head to head trials.”
BackgroundA controlled trial of sustained-release bupropion, a nicotine patch, or both for smoking cessation
Jorenby, D. E., Leischow, S. J., Nides, M. A., Rennard, S. I., Johnston, J. A., Hughes, A. R., Smith, S. S., Muramoto, M. L., Daughton, D. M., Doan, K., et al. (1999) · New England Journal of Medicine · doi:10.1056/nejm199903043400903 · Cited by 1,585
Network meta‐analysis for indirect treatment comparisons
Lumley, T. (2002) · Statistics in Medicine · doi:10.1002/sim.1201 · Cited by 1,206
Clinical trials: A practical approach
Daly, M. J. (1984) · Trends in Pharmacological Sciences · doi:10.1016/0165-6147(84)90515-7 · Cited by 1,053
Cited by
Graphical tools for network meta-analysis in STATA
Chaimani, A., Higgins, J. P. T., Mavridis, D., Spyridonos, P., & Salanti, G. (2013) · PLoS ONE · doi:10.1371/journal.pone.0076654 · Cited by 2,473
Puhan, M. A., Schunemann, H. J., Murad, M. H., Li, T., Brignardello-Petersen, R., Singh, J. A., Kessels, A. G., Guyatt, G. H., & for the GRADE Working Group (2014) · BMJ · doi:10.1136/bmj.g5630 · Cited by 1,812
GRADE guidelines: 8. Rating the quality of evidence—indirectness
Guyatt, G. H., Oxman, A. D., Kunz, R., Woodcock, J., Brozek, J., Helfand, M., Alonso-Coello, P., Falck-Ytter, Y., Jaeschke, R., Vist, G., et al. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2011.04.014 · Cited by 1,843
Salanti, G. (2012) · Research Synthesis Methods · doi:10.1002/jrsm.1037 · Cited by 1,729
How to increase value and reduce waste when research priorities are set
Chalmers, I., Bracken, M. B., Djulbegovic, B., Garattini, S., Grant, J., Gülmezoglu, A. M., Howells, D. W., Ioannidis, J. P. A., & Oliver, S. (2014) · The Lancet · doi:10.1016/s0140-6736(13)62229-1 · Cited by 1,436
Dias, S., Sutton, A. J., Ades, A. E., & Welton, N. J. (2013) · Medical Decision Making · doi:10.1177/0272989x12458724 · Cited by 1,207
White, I. R., Barrett, J. K., Jackson, D., & Higgins, J. P. T. (2012) · Research Synthesis Methods · doi:10.1002/jrsm.1045 · Cited by 1,114
Conceptual and technical challenges in network meta-analysis
Cipriani, A., Higgins, J. P. T., Geddes, J. R., & Salanti, G. (2013) · Annals of Internal Medicine · doi:10.7326/0003-4819-159-2-201307160-00008 · Cited by 1,072
“This assumption has been described alongside several terms in the literature, including similarity (14, 15), transitivity (16, 17), consistency (4), and coherence (18).”
BackgroundJansen, J. P., Fleurence, R., Devine, B., Itzler, R., Barrett, A., Hawkins, N., Lee, K., Boersma, C., Annemans, L., & Cappelleri, J. C. (2011) · Value in Health · doi:10.1016/j.jval.2011.04.002 · Cited by 1,036
Hoaglin, D. C., Hawkins, N., Jansen, J. P., Scott, D. A., Itzler, R., Cappelleri, J. C., Boersma, C., Thompson, D., Larholt, K. M., Diaz, M., et al. (2011) · Value in Health · doi:10.1016/j.jval.2011.01.011 · Cited by 770
Dias, S., Welton, N. J., Sutton, A. J., Caldwell, D. M., Lu, G., & Ades, A. E. (2013) · Medical Decision Making · doi:10.1177/0272989x12455847 · Cited by 728
Jansen, J. P., & Naci, H. (2013) · BMC Medicine · doi:10.1186/1741-7015-11-159 · Cited by 656
Risk of Clostridium difficile Infection With Acid Suppressing Drugs and Antibiotics: Meta-Analysis
Kwok, C. S., Arthur, A. K., Anibueze, C. I., Singh, S., Cavallazzi, R., & Loke, Y. K. (2012) · American Journal of Gastroenterology · doi:10.1038/ajg.2012.108 · Cited by 581
Self-citationEvaluation of inconsistency in networks of interventions
Veroniki, A. A., Vasiliadis, H. S., Higgins, J. P., & Salanti, G. (2013) · International Journal of Epidemiology · doi:10.1093/ije/dys222 · Cited by 576
GetReal in network meta-analysis: a review of the methodology
Efthimiou, O., Debray, T. P. A., van Valkenhoef, G., Trelle, S., Panayidou, K., Moons, K. G. M., Reitsma, J. B., Shang, A., Salanti, G., & on behalf of GetReal Methods Review Group (2016) · Research Synthesis Methods · doi:10.1002/jrsm.1195 · Cited by 401
Network meta-analysis-highly attractive but more methodological research is needed
Li, T., Puhan, M. A., Vedula, S. S., Singh, S., Dickersin, K., & The Ad Hoc Network Meta-analysis Methods Meeting Working Group (2011) · BMC Medicine · doi:10.1186/1741-7015-9-79 · Cited by 361
BUGSnet: an R package to facilitate the conduct and reporting of Bayesian network Meta-analyses
Béliveau, A., Boyne, D. J., Slater, J., Brenner, D., & Arora, P. (2019) · BMC Medical Research Methodology · doi:10.1186/s12874-019-0829-2 · Cited by 312
A design‐by‐treatment interaction model for network meta‐analysis with random inconsistency effects
Jackson, D., Barrett, J. K., Rice, S., White, I. R., & Higgins, J. P. T. (2014) · Statistics in Medicine · doi:10.1002/sim.6188 · Cited by 304
Nikolakopoulou, A., White, I. R., & Salanti, G. (2020) · Handbook of Meta-Analysis · doi:10.1201/9781315119403-10 · Cited by 303
Hauser, W., Petzke, F., Uceyler, N., & Sommer, C. (2011) · Rheumatology · doi:10.1093/rheumatology/keq354 · Cited by 290
Lip, G. Y. H., Larsen, T. B., Skjøth, F., & Rasmussen, L. H. (2012) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2012.03.019 · Cited by 288
Assessing key assumptions of network meta‐analysis: a review of methods
Donegan, S., Williamson, P., D'Alessandro, U., & Tudur Smith, C. (2013) · Research Synthesis Methods · doi:10.1002/jrsm.1085 · Cited by 267
Cochrane Reviews on the Benefits/Risks of Fluoride Toothpastes
Wong, M. C. M., Clarkson, J., Glenny, A.-M., Lo, E. C. M., Marinho, V. C. C., Tsang, B. W. K., Walsh, T., & Worthington, H. V. (2011) · Journal of Dental Research · doi:10.1177/0022034510393346 · Cited by 267
Self-citationYuan, J., Zhang, R., Yang, Z., Lee, J., Liu, Y., Tian, J., Qin, X., Ren, Z., Ding, H., Chen, Q., et al. (2013) · European Urology · doi:10.1016/j.eururo.2013.01.012 · Cited by 260
Padwal, R., Klarenbach, S., Wiebe, N., Birch, D., Karmali, S., Manns, B., Hazel, M., Sharma, A. M., & Tonelli, M. (2011) · Obesity Reviews · doi:10.1111/j.1467-789x.2011.00866.x · Cited by 260
Reporting of results from network meta-analyses: methodological systematic review
Bafeta, A., Trinquart, L., Seror, R., & Ravaud, P. (2014) · BMJ · doi:10.1136/bmj.g1741 · Cited by 220
Sample size and power considerations in network meta-analysis
Thorlund, K., & Mills, E. J. (2012) · Systematic Reviews · doi:10.1186/2046-4053-1-41 · Cited by 197
Geissbühler, M., Hincapié, C. A., Aghlmandi, S., Zwahlen, M., Jüni, P., & da Costa, B. R. (2021) · BMC Medical Research Methodology · doi:10.1186/s12874-021-01310-0 · Cited by 195
Hutton, B., Salanti, G., Chaimani, A., Caldwell, D. M., Schmid, C., Thorlund, K., Mills, E., Catalá-López, F., Turner, L., Altman, D. G., et al. (2014) · PLoS ONE · doi:10.1371/journal.pone.0092508 · Cited by 105
“Song et al [15] (2009) reviewed 88 systematic reviews published between 2000 and 2007 that involved the estimation of an indirect comparison in the form of either an adjusted indirect comparison, a frequentist/bayesian network meta-analysis, an informal indirect comparison, or a naive indirect comparison.”
BackgroundMethodSelf-citationLi, K.-P., Yang, X.-S., & Wu, T. (2022) · Frontiers in Endocrinology · doi:10.3389/fendo.2022.810242 · Cited by 79
Metadata
- Authors
- Song, F., Loke, Y. K., Walsh, T., Glenny, A.-M., Eastwood, A. J., & Altman, D. G.
- Venue
- BMJ
- Publisher
- BMJ
- Published
- 2009-04-03
- Volume
- 338
- Issue
- apr03 1
- Pages
- b1147-b1147
- ISSN
- 0007-1447 (ISSN-L)
- 0959-8138
- 1468-5833
- OpenAlex
- W2149012794
- Cited by
- 395
- References
- 21
- FWCI
- 2.53
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.