The DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index.
Methodological problems in the use of indirect comparisons for evaluating healthcare interventions: survey of published systematic reviews
Open accessAbstract
OBJECTIVE: To investigate basic assumptions and other methodological problems in the application of indirect comparison in systematic reviews of competing healthcare interventions. DESIGN: Survey of published systematic reviews. Inclusion criteria Systematic reviews published between 2000 and 2007 in which an indirect approach had been explicitly used. DATA EXTRACTION: Identified reviews were assessed for comprehensiveness of the literature search, method for indirect comparison, and whether assumptions about similarity and consistency were explicitly mentioned. RESULTS: The survey included 88 review reports. In 13 reviews, indirect comparison was informal. Results from different trials were naively compared without using a common control in six reviews. Adjusted indirect comparison was usually done using classic frequentist methods (n=49) or more complex methods (n=18). The key assumption of trial similarity was explicitly mentioned in only 40 of the 88 reviews. The consistency assumption was not explicit in most cases where direct and indirect evidence were compared or combined (18/30). Evidence from head to head comparison trials was not systematically searched for or not included in nine cases. CONCLUSIONS: Identified methodological problems were an unclear understanding of underlying assumptions, inappropriate search and selection of relevant trials, use of inappropriate or flawed methods, lack of objective and validated methods to assess or improve trial similarity, and inadequate comparison or inappropriate combination of direct and indirect evidence. Adequate understanding of basic assumptions underlying indirect and mixed treatment comparison is crucial to resolve these methodological problems. APPENDIX 1: PubMed search strategy. APPENDIX 2: Characteristics of identified reports. APPENDIX 3: Identified studies. References of included studies.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Meta-analysis in clinical trials
DerSimonian, R., & Laird, N. (1986) · Controlled Clinical Trials · doi:10.1016/0197-2456(86)90046-2 · Cited by 39,552
The combination of estimates from different experiments
Cochran, W. G. (1954) · Biometrics · doi:10.2307/3001666 · Cited by 5,380
Interaction revisited: the difference between two estimates.
Altman, D. G., & Bland, J. M. (2003) · BMJ · doi:10.1136/bmj.326.7382.219 · Cited by 3,203
Self-citationClinical Trials: A Practical Approach.
Hill, I. D., & Pocock, S. J. (1984) · Biometrics · doi:10.2307/2531186 · Cited by 2,930
Combination of direct and indirect evidence in mixed treatment comparisons
Lu, G., & Ades, A. E. (2004) · Statistics in Medicine · doi:10.1002/sim.1875 · Cited by 2,166
Bucher, H. C., Guyatt, G. H., Griffith, L. E., & Walter, S. D. (1997) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(97)00049-8 · Cited by 2,052
Simultaneous comparison of multiple treatments: Combining direct and indirect evidence.
Caldwell, D. M., Ades, A. E., & Higgins, J. P. T. (2005) · BMJ · doi:10.1136/bmj.331.7521.897 · Cited by 1,600 · cited 1×
“To improve statistical power evidence generated by indirect comparison can be combined with evidence from head to head trials.”
BackgroundA controlled trial of sustained-release bupropion, a nicotine patch, or both for smoking cessation
Jorenby, D. E., Leischow, S. J., Nides, M. A., Rennard, S. I., Johnston, J. A., Hughes, A. R., Smith, S. S., Muramoto, M. L., Daughton, D. M., Doan, K., et al. (1999) · New England Journal of Medicine · doi:10.1056/nejm199903043400903 · Cited by 1,585
Network meta‐analysis for indirect treatment comparisons
Lumley, T. (2002) · Statistics in Medicine · doi:10.1002/sim.1201 · Cited by 1,206
Clinical trials: A practical approach
Daly, M. J. (1984) · Trends in Pharmacological Sciences · doi:10.1016/0165-6147(84)90515-7 · Cited by 1,053
Cited by
Graphical tools for network meta-analysis in STATA
Chaimani, A., Higgins, J. P. T., Mavridis, D., Spyridonos, P., & Salanti, G. (2013) · PLoS ONE · doi:10.1371/journal.pone.0076654 · Cited by 2,473
Puhan, M. A., Schunemann, H. J., Murad, M. H., Li, T., Brignardello-Petersen, R., Singh, J. A., Kessels, A. G., Guyatt, G. H., & for the GRADE Working Group (2014) · BMJ · doi:10.1136/bmj.g5630 · Cited by 1,812
GRADE guidelines: 8. Rating the quality of evidence—indirectness
Guyatt, G. H., Oxman, A. D., Kunz, R., Woodcock, J., Brozek, J., Helfand, M., Alonso-Coello, P., Falck-Ytter, Y., Jaeschke, R., Vist, G., et al. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2011.04.014 · Cited by 1,843
Salanti, G. (2012) · Research Synthesis Methods · doi:10.1002/jrsm.1037 · Cited by 1,729
How to increase value and reduce waste when research priorities are set
Chalmers, I., Bracken, M. B., Djulbegovic, B., Garattini, S., Grant, J., Gülmezoglu, A. M., Howells, D. W., Ioannidis, J. P. A., & Oliver, S. (2014) · The Lancet · doi:10.1016/s0140-6736(13)62229-1 · Cited by 1,436
Metadata
- Authors
- Song, F., Loke, Y. K., Walsh, T., Glenny, A.-M., Eastwood, A. J., & Altman, D. G.
- Venue
- BMJ
- Publisher
- BMJ
- Published
- 2009-04-03
- Volume
- 338
- Issue
- apr03 1
- Pages
- b1147-b1147
- ISSN
- 0007-1447 (ISSN-L)
- 0959-8138
- 1468-5833
- OpenAlex
- W2149012794
- Cited by
- 395
- References
- 21
- FWCI
- 2.53
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Methodological problems in the use of indirect comparisons for evaluating healthcare interventions: survey of published systematic reviews. https://app.cochat.ai/papers/10.1136/bmj.b1147
- The paper
- Song, F., Loke, Y. K., Walsh, T., Glenny, A.-M., Eastwood, A. J., & Altman, D. G. (2009). Methodological problems in the use of indirect comparisons for evaluating healthcare interventions: survey of published systematic reviews. BMJ. https://doi.org/10.1136/bmj.b1147
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.