How often can meta-analyses of individual-level data individualize treatment? A meta-epidemiologic study
Open accessThe DOI matches its Crossref record; 55 of the 56 references it cites were found in a registry or an index.
Abstract
BACKGROUND: One of the claimed main advantages of individual participant data meta-analysis (IPDMA) is that it allows assessment of subgroup effects based on individual-level participant characteristics, and eventually stratified medicine. In this study, we evaluated the conduct and results of subgroup analyses in IPDMA. METHODS: We searched PubMed, EMBASE and the Cochrane Library from inception to 31 December 2014. We included papers if they described an IPDMA based on randomized clinical trials that investigated a therapeutic intervention on human subjects and in which the meta-analysis was preceded by a systematic literature search. We extracted data items related to subgroup analysis and subgroup differences (subgroup-treatment interaction p < 0.05). RESULTS: Overall, 327 IPDMAs were eligible. A statistically significant subgroup-treatment interaction for the primary outcome was reported in 102 (36.6%) of 279 IPDMAs that reported at least one subgroup analysis. This corresponded to 187 different statistically significant subgroup-treatment interactions: 124 for an individual-level subgrouping variable (in 76 IPDMAs) and 63 for a group-level subgrouping variable (in 36 IPDMAs). Of the 187, only 7 (3.7%; 6 individual and 1 group-level subgrouping variables) had a large difference between strata (standardized effect difference d ≥ 0.8). Among the 124 individual-level statistically significant subgroup differences, the IPDMA authors claimed that 42 (in 21 IPDMAs) should lead to treating the subgroups differently. None of these 42 had d ≥ 0.8. CONCLUSIONS: Availability of individual-level data provides statistically significant interactions for relative treatment effects in about a third of IPDMAs. A modest number of these interactions may offer opportunities for stratified medicine decisions.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Even large individual-level meta-analyses may not identify effect modification in most medical interventions 248 .”
From Preventive psychiatry: a blueprint for improving the mental health of young people (2021) · cites this paper as [248]
Citations
Cites
Statistical Power Analysis for the Behavioral Sciences
Muller, K. (1989) · Technometrics · doi:10.1080/00401706.1989.10488618 · Cited by 99,167
Statistical power analysis for the behavioral sciences
(1990) · Computers, Environment and Urban Systems · doi:10.1016/0198-9715(90)90050-4 · Cited by 65,269
Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000100 · Cited by 28,164
Self-citationCollaborators, C. T. T. (. (2012) · The Lancet · doi:10.1016/s0140-6736(12)60367-5 · Cited by 2,833
Emberson, J., Lees, K. R., Lyden, P., Blackwell, L., Albers, G., Bluhmki, E., Brott, T., Cohen, G., Davis, S., Donnan, G., et al. (2014) · The Lancet · doi:10.1016/s0140-6736(14)60584-5 · Cited by 2,737
(2004) · The Lancet · doi:10.1016/s0140-6736(04)15692-4 · Cited by 2,557
Antidepressant Drug Effects and Depression Severity: A Patient-Level Meta-analysis
Fournier, J. C., DeRubeis, R. J., Hollon, S. D., Dimidjian, S., Amsterdam, J. D., Shelton, R. C., & Fawcett, J. (2010) · JAMA · doi:10.1001/jama.2009.1943 · Cited by 2,009
Meta-analysis of individual participant data: rationale, conduct, and reporting
Riley, R. D., Lambert, P. C., & Abo-Zaid, G. (2010) · BMJ · doi:10.1136/bmj.c221 · Cited by 1,701
The Mass Production of Redundant, Misleading, and Conflicted Systematic Reviews and Meta‐analyses
IOANNIDIS, J. P. A. (2016) · The Milbank Quarterly · doi:10.1111/1468-0009.12210 · Cited by 1,483
Self-citationSubgroup analysis and other (mis)uses of baseline data in clinical trials
Assmann, S. F., Pocock, S. J., Enos, L. E., & Kasten, L. E. (2000) · The Lancet · doi:10.1016/s0140-6736(00)02039-0 · Cited by 1,167
Subgroup analysis in randomised controlled trials: importance, indications, and interpretation
Rothwell, P. M. (2005) · The Lancet · doi:10.1016/s0140-6736(05)17709-5 · Cited by 942
Hlatky, M. A., Boothroyd, D. B., Bravata, D. M., Boersma, E., Booth, J., Brooks, M. M., Carrié, D., Clayton, T. C., Danchin, N., Flather, M., et al. (2009) · The Lancet · doi:10.1016/s0140-6736(09)60552-3 · Cited by 735
Hills, R. K., Castaigne, S., Appelbaum, F. R., Delaunay, J., Petersdorf, S., Othus, M., Estey, E. H., Dombret, H., Chevret, S., Ifrah, N., et al. (2014) · The Lancet Oncology · doi:10.1016/s1470-2045(14)70281-5 · Cited by 721
Kotecha, D., Holmes, J., Krum, H., Altman, D. G., Manzano, L., Cleland, J. G. F., Lip, G. Y. H., Coats, A. J. S., Andersson, B., Kirchhof, P., et al. (2014) · The Lancet · doi:10.1016/s0140-6736(14)61373-8 · Cited by 579
Stewart, L. A., & Tierney, J. F. (2002) · Evaluation & the Health Professions · doi:10.1177/0163278702025001006 · Cited by 571
Franzosi, M. G., Santoro, E., Zuanetti, G., Baigent, C., Collins, R., Flather, M. D., Kjekshus, J., Latini, R., Liu, L., Maggioni, A. P., et al. (1998) · Circulation · Cited by 544
Artesunate combinations for treatment of malaria: meta-analysis
(2004) · The Lancet · doi:10.1016/s0140-6736(03)15162-8 · Cited by 538
Kent, D. M., & Hayward, R. A. (2007) · JAMA · doi:10.1001/jama.298.10.1209 · Cited by 505
Solomon, S. D., Wittes, J., Finn, P. V., Fowler, R., Viner, J., Bertagnolli, M. M., Arber, N., Levin, B., Meinert, C. L., Martin, B., et al. (2008) · Circulation · doi:10.1161/circulationaha.108.764530 · Cited by 367
Get real in individual participant data (IPD) meta‐analysis: a review of the methodology
Debray, T. P. A., Moons, K. G. M., van Valkenhoef, G., Efthimiou, O., Hummel, N., Groenwold, R. H. H., Reitsma, J. B., & on behalf of the GetReal methods review group (2015) · Research Synthesis Methods · doi:10.1002/jrsm.1160 · Cited by 348
Antibiotics for acute otitis media: a meta-analysis with individual patient data
Rovers, M. M., Glasziou, P., Appelman, C. L., Burke, P., McCormick, D. P., Damoiseaux, R. A., Gaboury, I., Little, P., & Hoes, A. W. (2006) · The Lancet · doi:10.1016/s0140-6736(06)69606-2 · Cited by 333
Troughton, R. W., Frampton, C. M., Brunner-La Rocca, H.-P., Pfisterer, M., Eurlings, L. W. M., Erntell, H., Persson, H., O'Connor, C. M., Moertl, D., Karlstrom, P., et al. (2014) · European Heart Journal · doi:10.1093/eurheartj/ehu090 · Cited by 275
Carotid endarterectomy for symptomatic carotid stenosis
Orrapin, S., & Rerkasem, K. (2017) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd001081.pub3 · Cited by 238
Postoperative radiotherapy for non-small cell lung cancer
Burdett, S., Rydzewska, L., Tierney, J., Fisher, D., Parmar, M. K., Arriagada, R., Pignon, J. P., Le Pechoux, C., & on behalf of the PORT Meta-analysis Trialists Group (2016) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd002142.pub4 · Cited by 200
Ioannidis, J. (2017) · British Journal of Sports Medicine · doi:10.1136/bjsports-2017-097621 · Cited by 188
Self-citationWallach, J. D., Sullivan, P. G., Trepanowski, J. F., Sainani, K. L., Steyerberg, E. W., & Ioannidis, J. P. A. (2017) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2016.9125 · Cited by 148
Self-citationMbuagbaw, L., van der Kop, M. L., Lester, R. T., Thirumurthy, H., Pop-Eleches, C., Ye, C., Smieja, M., Dolovich, L., Mills, E. J., & Thabane, L. (2013) · BMJ Open · doi:10.1136/bmjopen-2013-003950 · Cited by 147
Varadhan, R., Segal, J. B., Boyd, C. M., Wu, A. W., & Weiss, C. O. (2013) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2013.02.009 · Cited by 146
A decade of individual participant data meta-analyses: A review of current practice
Simmonds, M. C., Stewart, G. B., & Stewart, L. A. (2015) · Contemporary Clinical Trials · doi:10.1016/j.cct.2015.06.012 · Cited by 138
Szczech, L. A. M., Berlin, J. A., & Feldman, H. I. (1998) · Annals of Internal Medicine · doi:10.7326/0003-4819-128-10-199805150-00004 · Cited by 136
Carotid endarterectomy for symptomatic carotid stenosis
Rerkasem, K., & Rothwell, P. M. (2011) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd001081.pub2 · Cited by 119
Boonacker, C. W., Rovers, M. M., Browning, G. G., Hoes, A. W., Schilder, A. G., & Burton, M. J. (2014) · Health Technology Assessment · doi:10.3310/hta18050 · Cited by 117
Grommets in otitis media with effusion: an individual patient data meta-analysis
Rovers, M. M. (2005) · Archives of Disease in Childhood · doi:10.1136/adc.2004.059444 · Cited by 116
Greb, A., Bohlius, J., Schiefer, D., Schwarzer, G., Schulz, H., & Engert, A. (2008) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd004024.pub2 · Cited by 114
Nevitt, S. J., Marson, A. G., Davie, B., Reynolds, S., Williams, L., & Smith, C. T. (2017) · BMJ · doi:10.1136/bmj.j1390 · Cited by 93
Valgimigli, M., Sabaté, M., Kaiser, C., Brugaletta, S., Hernández, J., Galatius, S., Cequier, A. R., Eberli, F. R., de Belder, A., Serruys, P. W. J. C., et al. (2014) · BMJ · doi:10.1136/bmj.g6427 · Cited by 92
Goodacre, S. W., Stevens, J., Pandor, A., Poku, E. N., Ren, S., Cantrell, A. J., Bounes, V., Mas, A., Payen, D. M., Petrie, D. A., et al. (2014) · Academic Emergency Medicine · doi:10.1111/acem.12466 · Cited by 91
Wallach, J. D., Sullivan, P. G., Trepanowski, J. F., Steyerberg, E. W., & Ioannidis, J. P. A. (2016) · BMJ · doi:10.1136/bmj.i5826 · Cited by 89
Self-citationSharing Individual Patient Data from Clinical Trials
Drazen, J. M. (2015) · New England Journal of Medicine · doi:10.1056/nejmp1415160 · Cited by 77
POSTOPERATIVE RADIOTHERAPY OF NON-SMALL CELL LUNG CANCER
Фанисович, М. Ф., & Sharipova, N. S. (2015) · Malignant tumours · doi:10.18027/2224-5057-2014-3-91-96 · Cited by 77
Jorgensen, A., Alfirević, Ž., Smith, C. T., Williamson, P. R., & cerclage IPD Meta‐analysis Group (2007) · BJOG An International Journal of Obstetrics & Gynaecology · doi:10.1111/j.1471-0528.2007.01515.x · Cited by 75
Giallauria, F., Vigorito, C., Piepoli, M. F., & Coats, A. J. S. (2014) · International Journal of Cardiology · doi:10.1016/j.ijcard.2014.06.005 · Cited by 68
Vale, C. L., Rydzewska, L. H. M., Rovers, M. M., Emberson, J. R., Gueyffier, F., Stewart, L. A., & on behalf of the Cochrane IPD Meta-analysis Methods Group (2015) · BMJ · doi:10.1136/bmj.h1088 · Cited by 66
Data sharing statements for clinical trials
Taichman, D. B., Sahni, P., Pinborg, A., Peiperl, L., Laine, C., James, A., Hong, S.-T., Haileamlak, A., Gollogly, L., Godlee, F., et al. (2017) · BMJ · doi:10.1136/bmj.j2372 · Cited by 62
Individual patient data meta‐analysis of randomized anti‐epileptic drug monotherapy trials
Williamson, P. R., Marson, A. G., Smith, C. T., Hutton, J. L., & Chadwick, D. W. (2000) · Journal of Evaluation in Clinical Practice · doi:10.1046/j.1365-2753.2000.00236.x · Cited by 47
Meta-analyses Can Be Credible and Useful: A New Standard
Ioannidis, J. P. A. (2017) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2017.0035 · Cited by 36
Self-citationMeta-analyses triggered by previous (false-)significant findings: problems and solutions
Schuit, E., Roes, K. C. B., Mol, B. W., Kwee, A., Moons, K. G. M., & Groenwold, R. H. H. (2015) · Systematic Reviews · doi:10.1186/s13643-015-0048-9 · Cited by 36
Self-citationHuang, Y., Tang, J.-L., Tam, W., Mao, C., Yuan, J.-Q., Di, M., & Yang, Z. (2016) · Medicine · doi:10.1097/md.0000000000003312 · Cited by 35
Daniels, J. P., Middleton, L., Xiong, T., Champaneria, R., Johnson, N. P., Lichten, E. M., Sutton, C., Vercellini, P., Gray, R., Hills, R. K., et al. (2010) · Human Reproduction Update · doi:10.1093/humupd/dmq031 · Cited by 34
Efficacy of treating hypertension in women
the INDANA Investigators, Quan, A., Kerlikowske, K. M., Gueyffier, F., & Boissel, J. (1999) · Journal of General Internal Medicine · doi:10.1046/j.1525-1497.1999.12198.x · Cited by 33
Distribution and Epidemiological Characteristics of Published Individual Patient Data Meta-Analyses
Huang, Y., Mao, C., Yuan, J.-Q., Yang, Z., Di, M., Tam, W., & Tang, J.-L. (2014) · PLoS ONE · doi:10.1371/journal.pone.0100151 · Cited by 16
A simple method for converting an odds ratio to effect size for use in meta-analysis
Chinn, S. (2000) · Statistics in Medicine · doi:10.1002/1097-0258(20001130)19:22<3127::aid-sim784>3.0.co;2-m
Cited by
Schandelmaier, S., Briel, M., Varadhan, R., Schmid, C. H., Devasenapathy, N., Hayward, R. A., Gagnier, J., Borenstein, M., van der Heijden, G. J. M. G., Dahabreh, I. J., et al. (2020) · Canadian Medical Association Journal · doi:10.1503/cmaj.200077 · Cited by 749
Self-citationPreventive psychiatry: a blueprint for improving the mental health of young people
Fusar‐Poli, P., Correll, C. U., Arango, C., Berk, M., Patel, V., & Ioannidis, J. P. A. (2021) · World Psychiatry · doi:10.1002/wps.20869 · Cited by 497
Self-citationKent, D. M., van Klaveren, D., Paulus, J. K., D'Agostino, R., Goodman, S., Hayward, R., Ioannidis, J. P. A., Patrick-Lake, B., Morton, S., Pencina, M., et al. (2020) · Annals of Internal Medicine · doi:10.7326/m18-3668 · Cited by 179
Self-citationRiley, R. D., Debray, T. P. A., Fisher, D., Hattle, M., Marlin, N., Hoogland, J., Gueyffier, F., Staessen, J. A., Wang, J., Moons, K. G. M., et al. (2020) · Statistics in Medicine · doi:10.1002/sim.8516 · Cited by 175
Korang, S. K., Juul, S., Nielsen, E. E., Feinberg, J. B., Siddiqui, F., Ong, G., Klingenberg, S. L., Veroniki, A. A., Bu, F.-L., Thabane, L., et al. (2020) · Systematic Reviews · doi:10.1186/s13643-020-01516-1 · Cited by 46
Metadata
- Authors
- Schuit, E., Li, A. H., & Ioannidis, J. P. A.
- Affiliations
- Utrecht University (NL)
- Ottawa Hospital Research Institute (CA)
- Stanford University (US)
- Corresponding author
- Schuit, Ewoud
- Venue
- International Journal of Epidemiology
- Publisher
- Oxford University Press (OUP)
- Published
- 2018-11-15
- Volume
- 48
- Issue
- 2
- Pages
- 596-608
- ISSN
- 0300-5771 (ISSN-L)
- 1464-3685
- OpenAlex
- W2901826044
- Cited by
- 27
- References
- 56
- Citations by year
- 2026: 3 · 2025: 2 · 2024: 2 · 2023: 2 · 2022: 1 · 2021: 3 · 2020: 10 · 2019: 4
- FWCI
- 0.84
- Type
- journal article
- Language
- en
- Open access
- green
- dspace.library.uu.nl
- Licence
- Locations
- International Journal of Epidemiology (publishedVersion)
- PubMed (publishedVersion)
- Utrecht University Repository (Utrecht University) (submittedVersion, other-oa)
- Utrecht University Repository (Utrecht University) (submittedVersion)
- Funders
- Netherlands Organisation for Scientific Research (825.14.001)
- Laura and John Arnold Foundation
- Sue and Bob O’Donnell
- Canadian Institute for Health Research Doctoral Scholarship
- Michael Smith Foreign Study Supplement
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Keywords
- individual patient data meta-analysis, subgroup analysis, stratified medicine, microdata, treatment effect heterogeneity, meta-epidemiology
- MeSH terms
- Data Collection, Humans, Research Design, Meta-Analysis as Topic, Epidemiologic Studies
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.