The relation between treatment benefit and underlying risk in meta-analysis
Open accessThe DOI matches its Crossref record; 7 of the 7 references it cites were found in a registry or an index.
Abstract
In meta-analyses of clinical trials comparing a treated group with a control group it has been common to ask whether the treatment benefit varies according to the underlying risk of the patients in the different trials, with the hope of defining which patients would benefit most and which least from medical interventions. The usual analysis used to investigate this issue, however, which uses the observed proportions of events in the control groups of the trials as a measure of the underlying risk, is flawed and produces seriously misleading results. This arises through a bias due to regression to the mean and will be particularly acute in meta-analyses which include some small trials or in which the variability in the true underlying risks across trials is small. Approaches which previously have been thought to be more appropriate are to substitute the average proportion of events in the control and treated groups as the measure of underlying risk or to plot the proportion of events in the treated group against that in the control group (L'Abbé plot). However, these are still subject to bias in most circumstances. Because of the potentially seriously flawed conclusions that can result from such analyses, they should be replaced either by statistically appropriate (but more complex) approaches or, preferably, by analyses which investigate the dependence of the treatment effect on measured baseline characteristics of the patients in each trial.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Comparing methods of measurement: why plotting difference against standard method is misleading
Bland, J. M., & Altman, D. G. (1995) · The Lancet · doi:10.1016/s0140-6736(95)91748-9 · Cited by 2,343
Self-citationMeta-Analysis in Clinical Research
L'ABBÉ, K. A., DETSKY, A. S., & O'ROURKE, K. (1987) · Annals of Internal Medicine · doi:10.7326/0003-4819-107-2-224 · Cited by 1,267
Systematic Review: Why sources of heterogeneity in meta-analysis should be investigated
Thompson, S. G. (1994) · BMJ · doi:10.1136/bmj.309.6965.1351 · Cited by 1,017
Statistic Notes: Regression towards the mean
Bland, J. M., & Altman, D. G. (1994) · BMJ · doi:10.1136/bmj.308.6942.1499 · Cited by 547
Self-citationA bivariate approach to meta-analysis
Van Houwelingen, H. C., Zwinderman, K. H., & Stijnen, T. (1993) · Statistics in Medicine · doi:10.1002/sim.4780122405 · Cited by 327
Who benefits from medical interventions?
Smith, G. D., & Egger, M. (1994) · BMJ · doi:10.1136/bmj.308.6921.72 · Cited by 128
Cited by
Systematic reviews in the social sciences: a practical guide
(2006) · Choice Reviews Online · doi:10.5860/choice.43-5664 · Cited by 4,089
How should meta‐regression analyses be undertaken and interpreted?
Thompson, S. G., & Higgins, J. P. T. (2002) · Statistics in Medicine · doi:10.1002/sim.1187 · Cited by 3,026
Self-citationAdvanced methods in meta‐analysis: multivariate approach and meta‐regression
van Houwelingen, H. C., Arends, L. R., & Stijnen, T. (2002) · Statistics in Medicine · doi:10.1002/sim.1040 · Cited by 1,580
Thrombolysis for acute ischaemic stroke
Wardlaw, J. M., Murray, V., Berge, E., & del Zoppo, G. J. (2014) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd000213.pub3 · Cited by 1,370
Early enteral nutrition in acutely ill patients: a systematic review
Marik, P. E., & Zaloga, G. P. (2001) · Critical Care Medicine · doi:10.1097/00003246-200112000-00005 · Cited by 938
Statistical strategies for avoiding false discoveries in metabolomics and related experiments
Broadhurst, D. I., & Kell, D. B. (2007) · Metabolomics · doi:10.1007/s11306-006-0037-z · Cited by 817
Deeks, J. J. (2002) · Statistics in Medicine · doi:10.1002/sim.1188 · Cited by 660
“Several authors have noted the problem of investigating variation in measures of treatment e ect with control group event rate [30; 31], and developed methods for correctly estimating such relationships accounting for the correlation between control group event rates and treatment e ects [32–34].”
MethodProblems in the “Evidence” of “Evidence-Based Medicine”
Feinstein, A. R., & Horwitz, R. I. (1997) · The American Journal of Medicine · doi:10.1016/s0002-9343(97)00244-1 · Cited by 653
Summing up evidence: one answer is not always enough
Lau, J., Ioannidis, J. P., & Schmid, C. H. (1998) · The Lancet · doi:10.1016/s0140-6736(97)08468-7 · Cited by 655
Recent developments in meta‐analysis
Sutton, A. J., & Higgins, J. P. T. (2008) · Statistics in Medicine · doi:10.1002/sim.2934 · Cited by 590
Misleading funnel plot for detection of bias in meta-analysis
Tang, J.-L., & Liu, J. L. (2000) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(99)00204-8 · Cited by 420
Prediction of benefit from carotid endar terectomy in individual patients: a risk-modelling study
Rothwell, P., & Warlow, C. (1999) · The Lancet · doi:10.1016/s0140-6736(98)11415-0 · Cited by 411
Lambert, P. C., Sutton, A. J., Abrams, K. R., & Jones, D. R. (2002) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(01)00414-0 · Cited by 394
Tsapas, A., Avgerinos, I., Karagiannis, T., Malandris, K., Manolopoulos, A., Andreadis, P., Liakos, A., Matthews, D. R., & Bekiari, E. (2020) · Annals of Internal Medicine · doi:10.7326/m20-0864 · Cited by 346
Counsell, C., Dennis, M., McDowall, M., & Warlow, C. (2002) · Stroke · doi:10.1161/hs0402.105909 · Cited by 319
Furukawa, T. A., Guyatt, G. H., & Griffith, L. E. (2002) · International Journal of Epidemiology · doi:10.1093/ije/31.1.72 · Cited by 318
Numbers needed to treat derived from meta-analyses---sometimes informative, usually misleading
Smeeth, L., Haines, A., & Ebrahim, S. (1999) · BMJ · doi:10.1136/bmj.318.7197.1548 · Cited by 302
Systematic review of evidence on thrombolytic therapy for acute ischaemic stroke
Wardlaw, J., Warlow, C., & Counsell, C. (1997) · The Lancet · doi:10.1016/s0140-6736(97)03022-5 · Cited by 287
Warn, D. E., Thompson, S. G., & Spiegelhalter, D. J. (2002) · Statistics in Medicine · doi:10.1002/sim.1189 · Cited by 277
Self-citationMicroalbuminuria, blood pressure and diabetic renal disease: origin and development of ideas
Mogensen, C. E. (1999) · Diabetologia · doi:10.1007/s001250051151 · Cited by 274
Thrombolysis for acute ischaemic stroke
Wardlaw, J. M., Murray, V., Berge, E., & del Zoppo, G. J. (2009) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd000213.pub2 · Cited by 267
Dans, A. L., Dans, L. F., Guyatt, G. H., Richardson, S., & for the Evidence-Based Medicine Working Group (1998) · JAMA · doi:10.1001/jama.279.7.545 · Cited by 257
Issues in Comparisons Between Meta-analyses and Large Trials
Ioannidis, J. P. A. (1998) · JAMA · doi:10.1001/jama.279.14.1089 · Cited by 253
Methods for Exploring Heterogeneity in Meta-Analysis
Song, F., Sheldon, T. A., Sutton, A. J., Abrams, K. R., & Jones, D. R. (2001) · Evaluation & the Health Professions · doi:10.1177/01632780122034849 · Cited by 253
De Luca, G., Navarese, E., & Marino, P. (2009) · European Heart Journal · doi:10.1093/eurheartj/ehp118 · Cited by 252
Systematic Review of Prognostic Models in Patients with Acute Stroke
Counsell, C., & Dennis, M. (2001) · Cerebrovascular Diseases · doi:10.1159/000047699 · Cited by 250
Geissbühler, M., Hincapié, C. A., Aghlmandi, S., Zwahlen, M., Jüni, P., & da Costa, B. R. (2021) · BMC Medical Research Methodology · doi:10.1186/s12874-021-01310-0 · Cited by 195
Meta-analysis. Beyond the grand mean?
Smith, G. D., Egger, M., & Phillips, A. N. (1997) · BMJ · doi:10.1136/bmj.315.7122.1610 · Cited by 174
Can meta-analysis help target interventions at individuals most likely to benefit?
Thompson, S. G., & Higgins, J. P. (2005) · The Lancet · doi:10.1016/s0140-6736(05)70200-2 · Cited by 174
Self-citationSawyer, L. M., Malottki, K., Sabry-Grant, C., Yasmeen, N., Wright, E., Sohrt, A., Borg, E., & Warren, R. B. (2019) · PLOS ONE · doi:10.1371/journal.pone.0220868 · Cited by 161
Biondi-Zoccai, G. G. L., Lotrionte, M., Abbate, A., Testa, L., Remigi, E., Burzotta, F., Valgimigli, M., Romagnoli, E., Crea, F., & Agostoni, P. (2006) · BMJ · doi:10.1136/bmj.38693.516782.7c · Cited by 149
Metadata
- Authors
- Sharp, S. J., Thompson, S. G., & Altman, D. G.
- Venue
- BMJ
- Publisher
- BMJ
- Published
- 1996-09-21
- Volume
- 313
- Issue
- 7059
- Pages
- 735-738
- ISSN
- 0007-1447 (ISSN-L)
- 0959-8138
- 1468-5833
- OpenAlex
- W2056083953
- Cited by
- 172
- References
- 7
- FWCI
- 18.05
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.