Statistical Analysis of Individual Participant Data Meta-Analyses: A Comparison of Methods and Recommendations for Practice
Open accessThe DOI matches its Crossref record; 6 of the 6 references it cites were found in a registry or an index.
Abstract
BACKGROUND: Individual participant data (IPD) meta-analyses that obtain "raw" data from studies rather than summary data typically adopt a "two-stage" approach to analysis whereby IPD within trials generate summary measures, which are combined using standard meta-analytical methods. Recently, a range of "one-stage" approaches which combine all individual participant data in a single meta-analysis have been suggested as providing a more powerful and flexible approach. However, they are more complex to implement and require statistical support. This study uses a dataset to compare "two-stage" and "one-stage" models of varying complexity, to ascertain whether results obtained from the approaches differ in a clinically meaningful way. METHODS AND FINDINGS: We included data from 24 randomised controlled trials, evaluating antiplatelet agents, for the prevention of pre-eclampsia in pregnancy. We performed two-stage and one-stage IPD meta-analyses to estimate overall treatment effect and to explore potential treatment interactions whereby particular types of women and their babies might benefit differentially from receiving antiplatelets. Two-stage and one-stage approaches gave similar results, showing a benefit of using anti-platelets (Relative risk 0.90, 95% CI 0.84 to 0.97). Neither approach suggested that any particular type of women benefited more or less from antiplatelets. There were no material differences in results between different types of one-stage model. CONCLUSIONS: For these data, two-stage and one-stage approaches to analysis produce similar results. Although one-stage models offer a flexible environment for exploring model structure and are useful where across study patterns relating to types of participant, intervention and outcome mask similar relationships within trials, the additional insights provided by their usage may not outweigh the costs of statistical support for routine application in syntheses of randomised controlled trials. Researchers considering undertaking an IPD meta-analysis should not necessarily be deterred by a perceived need for sophisticated statistical methods when combining information from large randomised trials.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Conducting Meta-Analyses in R with the metafor Package
Viechtbauer, W. (2010) · Journal of Statistical Software · doi:10.18637/jss.v036.i03 · Cited by 20,464
Data Analysis Using Regression and Multilevel/Hierarchical Models
Gelman, A., & Hill, J. (2006) · doi:10.1017/cbo9780511790942 · Cited by 13,597
A Re-Evaluation of Random-Effects Meta-Analysis
Higgins, J. P. T., Thompson, S. G., & Spiegelhalter, D. J. (2009) · Journal of the Royal Statistical Society Series A: Statistics in Society · doi:10.1111/j.1467-985x.2008.00552.x · Cited by 2,569 · cited 1×
“For instance, over-fitting may occur when the number of studies is (relatively) small or the normality assumption of random-effects is violated [25].”
BackgroundPublication Bias in Meta‐Analysis: Prevention, Assessment and Adjustments
(2005) · doi:10.1002/0470870168 · Cited by 2,368
The BUGS project: evolution, critique and future directions
Lunn, D., Spiegelhalter, D., Thomas, A., & Best, N. (2009) · Statistics in Medicine · doi:10.1002/sim.3680 · Cited by 1,876
Meta-analysis of individual participant data: rationale, conduct, and reporting
Riley, R. D., Lambert, P. C., & Abo-Zaid, G. (2010) · BMJ · doi:10.1136/bmj.c221 · Cited by 1,701 · cited 1×
“A major advantage of IPD analysis over the conventional aggregate data approach is that it allows detailed participant-level exploration of treatment effectiveness in relation to individuals’ characteristics such as age or stage of disease [2,5].”
Method
Cited by
Karlas, T., Petroff, D., Sasso, M., Fan, J.-G., Mi, Y.-Q., de Lédinghen, V., Kumar, M., Lupsor-Platon, M., Han, K.-H., Cardoso, A. C., et al. (2017) · Journal of Hepatology · doi:10.1016/j.jhep.2016.12.022 · Cited by 1,275
“3) ALT c , U/L, median [IQR] female 47 [27, 83] 42 [24, 79] 47 [31, 81] 59 [35, 91] 58 [38, 98] male 56 [34, 94] 53 [31, 93] 49 [33, 79] 67 [45, 101] 87 [61, 135] AST d , U/L, median [IQR] female 42 [27, 69] 37 [25, 62] 44 [29, 75] 47 [31, 76] 51 [37, 78] male 40 [29, 62] 39 [28, 65] 38 [27, 56] 43 [33, 61] 51 [38, 71…”
BackgroundInfluentialBurke, D. L., Ensor, J., & Riley, R. D. (2017) · Statistics in Medicine · doi:10.1002/sim.7141 · Cited by 538
“97) analyses were almost identical [2,13].”
BackgroundMethodResultInfluentialBally, M., Dendukuri, N., Rich, B., Nadeau, L., Helin-Salmivaara, A., Garbe, E., & Brophy, J. M. (2017) · BMJ · doi:10.1136/bmj.j1909 · Cited by 528
Sahgal, A., Aoyama, H., Kocher, M., Neupane, B., Collette, S., Tago, M., Shaw, P., Beyene, J., & Chang, E. L. (2015) · International Journal of Radiation Oncology*Biology*Physics · doi:10.1016/j.ijrobp.2014.10.024 · Cited by 420
Gayet-Ageron, A., Prieto-Merino, D., Ker, K., Shakur, H., Ageron, F.-X., Roberts, I., Kayani, A., Geer, A., Ndungu, B., Fawole, B., et al. (2018) · The Lancet · doi:10.1016/s0140-6736(17)32455-8 · Cited by 396
Litton, E., Xiao, J., & Ho, K. M. (2013) · BMJ · doi:10.1136/bmj.f4822 · Cited by 394
Citizens, consumers and farm animal welfare: A meta-analysis of willingness-to-pay studies
Clark, B., Stewart, G. B., Panzone, L. A., Kyriazakis, I., & Frewer, L. J. (2017) · Food Policy · doi:10.1016/j.foodpol.2017.01.006 · Cited by 374
Self-citationTierney, J. F., Vale, C., Riley, R., Smith, C. T., Stewart, L., Clarke, M., & Rovers, M. (2015) · PLOS Medicine · doi:10.1371/journal.pmed.1001855 · Cited by 361
Self-citationGet real in individual participant data (IPD) meta‐analysis: a review of the methodology
Debray, T. P. A., Moons, K. G. M., van Valkenhoef, G., Efthimiou, O., Hummel, N., Groenwold, R. H. H., Reitsma, J. B., & on behalf of the GetReal methods review group (2015) · Research Synthesis Methods · doi:10.1002/jrsm.1160 · Cited by 348
“The two-stage approach can, however, still be useful to explore the available data, to present intermediate results or to identify key challenges when designing a one-stage meta-analytical model (Stewart et al., 2012).”
BackgroundInfluentialKaasch, A. J., Barlow, G., Edgeworth, J. D., Fowler, V. G., Hellmich, M., Hopkins, S., Kern, W. V., Llewelyn, M. J., Rieg, S., Rodriguez-Baño, J., et al. (2014) · Journal of Infection · doi:10.1016/j.jinf.2013.10.015 · Cited by 277
Inhaled Cannabis for Chronic Neuropathic Pain: A Meta-analysis of Individual Patient Data
Andreae, M. H., Carter, G. M., Shaparin, N., Suslov, K., Ellis, R. J., Ware, M. A., Abrams, D. I., Prasad, H., Wilsey, B., Indyk, D., et al. (2015) · The Journal of Pain · doi:10.1016/j.jpain.2015.07.009 · Cited by 266
Henderson, J. T., Vesco, K. K., Senger, C. A., Thomas, R. G., & Redmond, N. (2021) · JAMA · doi:10.1001/jama.2021.8551 · Cited by 227
Fixed- and random-effects meta-analytic structural equation modeling: Examples and analyses in R
Cheung, M. W.-L. (2014) · Behavior Research Methods · doi:10.3758/s13428-013-0361-y · Cited by 208
“In the area of medical sciences, the individual patient data meta-analysis—that is, the synthesis of raw data rather than summary statistics—is becoming popular (e.g., Riley, Lambert, & Abo-Zaid, 2010; Stewart et al., 2012; Sutton, Kendrick, & Coupland, 2008).”
BackgroundKasai, M., Cipriani, F., Gayet, B., Aldrighetti, L., Ratti, F., Sarmiento, J. M., Scatton, O., Kim, K.-H., Dagher, I., Topal, B., et al. (2018) · Surgery · doi:10.1016/j.surg.2018.01.020 · Cited by 202
Individual Participant Data Meta-Analysis for a Binary Outcome: One-Stage or Two-Stage?
Debray, T. P. A., Moons, K. G. M., Abo-Zaid, G. M. A., Koffijberg, H., & Riley, R. D. (2013) · PLoS ONE · doi:10.1371/journal.pone.0060650 · Cited by 201
“[37] conclude that ‘two-stage and one-stage approaches to analysis produce similar results’ and ‘where an IPD review evaluates effectiveness based on sufficient data from randomised controlled trials, one-stage statistical analyses may not add much value to simpler two-stage approaches’.”
MethodResultTypes of parental involvement in CBT with anxious youth: a preliminary meta-analysis
Manassis, K., Lee, T. C., Bennett, K., Zhao, X. Y., Mendlowitz, S., Duda, S., Saini, M., Wilansky, P., Baer, S., Barrett, P., et al. (2014) · Journal of Consulting and Clinical Psychology · doi:10.1037/a0036969 · Cited by 199
Flückiger, C., Rubel, J., Del Re, A. C., Horvath, A. O., Wampold, B. E., Crits-Christoph, P., Atzil-Slonim, D., Compare, A., Falkenström, F., Ekeblad, A., et al. (2020) · Journal of Consulting and Clinical Psychology · doi:10.1037/ccp0000594 · Cited by 180
“This analysis followed a two-stage individual participant data meta-analysis approach (e.g., Stewart et al., 2012; Tierney et al., 2015).”
MethodNagendran, M., Russell, J. A., Walley, K. R., Brett, S. J., Perkins, G. D., Hajjar, L., Mason, A. J., Ashby, D., & Gordon, A. C. (2019) · Intensive Care Medicine · doi:10.1007/s00134-019-05620-2 · Cited by 161
Mahmoodi, B. K., Cushman, M., Anne Næss, I., Allison, M. A., Bos, W. J., Brækkan, S. K., Cannegieter, S. C., Gansevoort, R. T., Gona, P. N., Hammerstrøm, J., et al. (2017) · Circulation · doi:10.1161/circulationaha.116.024507 · Cited by 158
Rozental, A., Magnusson, K., Boettcher, J., Andersson, G., & Carlbring, P. (2017) · Journal of Consulting and Clinical Psychology · doi:10.1037/ccp0000158 · Cited by 137
Sulis, G., Daniels, B., Kwan, A., Gandra, S., Daftary, A., Das, J., & Pai, M. (2020) · BMJ Global Health · doi:10.1136/bmjgh-2020-003393 · Cited by 132
Hua, H., Burke, D. L., Crowther, M. J., Ensor, J., Tudur Smith, C., & Riley, R. D. (2017) · Statistics in Medicine · doi:10.1002/sim.7171 · Cited by 113
Panagioti, M., Bower, P., Kontopantelis, E., Lovell, K., Gilbody, S., Waheed, W., Dickens, C., Archer, J., Simon, G., Ell, K., et al. (2016) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2016.1794 · Cited by 94
Pescott, O. L., & Stewart, G. B. (2014) · PeerJ · doi:10.7717/peerj.360 · Cited by 82
“…would also allow a more powerful analysis using a hierarchical modeling approach, with increased power to detect real differences between groups, improved ability to explore interaction within and between trials, and would also allow for formal explorations of model choice (Stewart et al. 2012).”
Self-citationMalaria, malnutrition, and birthweight: A meta-analysis using individual participant data
Cates, J. E., Unger, H. W., Briand, V., Fievet, N., Valea, I., Tinto, H., D’Alessandro, U., Landis, S. H., Adu-Afarwuah, S., Dewey, K. G., et al. (2017) · PLOS Medicine · doi:10.1371/journal.pmed.1002373 · Cited by 74
“We employed a 2-stage approach, as it is generally considered more easily interpretable and allows the investigator to visually present forest plots and quantify statistical heterogeneity [36].”
MethodGluud, L. L., Knop, F. K., & Vilsbøll, T. (2014) · BMJ Open · doi:10.1136/bmjopen-2014-005325 · Cited by 59
Persson, F., Lindhardt, M., Rossing, P., & Parving, H.-H. (2016) · Journal of the Renin-Angiotensin-Aldosterone System · doi:10.1177/1470320316652047 · Cited by 54
Lin, J., Scott, W., Carpenter, L., Norton, S., Domhardt, M., Baumeister, H., & McCracken, L. M. (2019) · Systematic Reviews · doi:10.1186/s13643-019-1044-2 · Cited by 30
“We will perform analyses according to current expert recommendations for individual participant data metaanalysis [45, 89].”
MethodRuthmann, F., Lo, J. W., Mendyk-Bordet, A.-M., Allart, E., Köhler, S., Klimkowicz-Mrowiec, A., Staals, J., Sachdev, P. S., Bordet, R., & Dondaine, T. (2025) · Journal of Affective Disorders · doi:10.1016/j.jad.2024.10.099 · Cited by 23
Metadata
- Authors
- Stewart, G. B., Altman, D. G., Askie, L. M., Duley, L., Simmonds, M. C., & Stewart, L. A.
- Venue
- PLoS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2012-10-03
- Volume
- 7
- Issue
- 10
- Pages
- e46042
- ISSN
- 1932-6203 (ISSN-L)
- OpenAlex
- W2043241676
- Cited by
- 232
- References
- 6
- FWCI
- 13.80
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.