The DOI matches its Crossref record; 6 of the 6 references it cites were found in a registry or an index.
Individual participant data meta-analyses compared with meta-analyses based on aggregate data
Open accessAbstract
BACKGROUND: Meta-analyses based on individual participant data (IPD-MAs) allow more powerful and uniformly consistent analyses as well as better characterisation of subgroups and outcomes, compared to those which are based on aggregate data (AD-MAs) extracted from published trial reports. However, IPD-MAs are a larger undertaking requiring greater resources than AD-MAs. Researchers have compared results from IPD-MA against results obtained from AD-MA and reported conflicting findings. We present a methodology review to summarise this empirical evidence . OBJECTIVES: To review systematically empirical comparisons of meta-analyses of randomised trials based on IPD with those based on AD extracted from published reports, to evaluate the level of agreement between IPD-MA and AD-MA and whether agreement is affected by differences in type of effect measure, trials and participants included within the IPD-MA and AD-MA, and whether analyses were undertaken to explore the main effect of treatment or a treatment effect modifier. SEARCH METHODS: An electronic search of the Cochrane Library (includes Cochrane Database of Systematic Reviews, Database of Abstracts of Reviews of Effectiveness, CENTRAL, Cochrane Methodology Register, HTA database, NHS Economic Evaluations Database), MEDLINE, and Embase was undertaken up to 7 January 2016. Potentially relevant articles that were known to any of the review authors and reference lists of retrieved articles were also checked. SELECTION CRITERIA: Studies reporting an empirical comparison of the results of meta-analyses of randomised trials using IPD with those using AD. Studies were included if sufficient numerical data, comparing IPD-MA and AD-MA, were available in their reports. DATA COLLECTION AND ANALYSIS: Two review authors screened the title and abstract of identified studies with full-text publications retrieved for those identified as eligible or potentially eligible. A 'quality' assessment was done and data were extracted independently by two review authors with disagreements resolved by involving a third author. Data were summarised descriptively for comparisons where an estimate of effect measure and corresponding precision have been provided both for IPD-MA and for AD-MA in the study report. Comparisons have been classified according to whether identical effect measures, identical trials and patients had been used in the IPD-MA and the AD-MA, and whether the analyses were undertaken to explore the main effect of treatment, or to explore a potential treatment effect modifier.Effect measures were transformed to a standardised scale (z scores) and scatter plots generated to allow visual comparisons. For each comparison, we compared the statistical significance (at the 5% two-sided level) of an IPD-MA compared to the corresponding AD-MA and calculated the number of discrepancies. We examined discrepancies by type of analysis (main effect or modifier) and according to whether identical trials, patients and effect measures had been used by the IPD-MA and AD-MA. We calculated the average of differences between IPD-MA and AD-MA (z scores, ratio effect estimates and standard errors (of ratio effects)) and 95% limits of agreement. MAIN RESULTS: From the 9330 reports found by our searches, 39 studies were eligible for this review with effect estimate and measure of precision extracted for 190 comparisons of IPD-MA and AD-MA. We classified the quality of studies as 'no important flaws' (29 (74%) studies) or 'possibly important flaws' (10 (26%) studies).A median of 4 (interquartile range (IQR): 2 to 6) comparisons were made per study, with 6 (IQR 4 to 11) trials and 1225 (542 to 2641) participants in IPD-MAs and 7 (4 to 11) and 1225 (705 to 2541) for the AD-MAs. One hundred and forty-four (76%) comparisons were made on the main treatment effect meta-analysis and 46 (24%) made using results from analyses to explore treatment effect modifiers.There is agreement in statistical significance between the IPD-MA and AD-MA for 152 (80%) comparisons, 23 of which disagreed in direction of effect. There is disagreement in statistical significance for 38 (20%) comparisons with an excess proportion of IPD-MA detecting a statistically significant result that was not confirmed with AD-MA (28 (15%)), compared with 10 (5%) comparisons with a statistically significant AD-MA that was not confirmed by IPD-MA. This pattern of disagreement is consistent for the 144 main effect analyses but not for the 46 comparisons of treatment effect modifier analyses. Conclusions from some IPD-MA and AD-MA differed even when based on identical trials, participants (but not necessarily identical follow-up) and treatment effect measures. The average difference between IPD-MA and AD-MA in z scores, ratio effect estimates and standard errors is small but limits of agreement are wide and include important differences in both directions. Discrepancies between IPD-MA and AD-MA do not appear to increase as the differences
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Rothwell, P. M., Fowkes, F. G. R., Belch, J. F., Ogawa, H., Warlow, C. P., & Meade, T. W. (2011) · The Lancet · doi:10.1016/s0140-6736(10)62110-1 · Cited by 1,497
CHALMERS, I. (1993) · Annals of the New York Academy of Sciences · doi:10.1111/j.1749-6632.1993.tb26345.x · Cited by 561
Ahmed, I., Sutton, A. J., & Riley, R. D. (2012) · BMJ · doi:10.1136/bmj.d7762 · Cited by 404
Self-citationCooper, H., & Patall, E. A. (2009) · Psychological Methods · doi:10.1037/a0015565 · Cited by 339
Michiels, S., Piedbois, P., Burdett, S., Syz, N., Stewart, L., & Pignon, J.-P. (2005) · International Journal of Technology Assessment in Health Care · doi:10.1017/s0266462305050154 · Cited by 154
Brouwer, I. A., Raitt, M. H., Dullemeijer, C., Kraemer, D. F., Zock, P. L., Morris, C., Katan, M. B., Connor, W. E., Camm, J. A., Schouten, E. G., et al. (2008) · European Heart Journal · doi:10.1093/eurheartj/ehp003 · Cited by 120
Cited by
Antiplatelet agents for preventing pre-eclampsia and its complications
Duley, L., Henderson-Smart, D. J., Meher, S., & King, J. F. (2007) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd004659.pub2 · Cited by 1,016
Antiplatelet agents for preventing pre-eclampsia and its complications
Duley, L., Meher, S., Hunter, K. E., Seidler, A. L., & Askie, L. M. (2019) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd004659.pub3 · Cited by 366
Dafni, U., Michielin, O., Lluesma, S. M., Tsourti, Z., Polydoropoulou, V, Karlis, D., Besser, M. J., Haanen, J., Svane, I. M., Ohashi, P. S., et al. (2019) · Annals of Oncology · doi:10.1093/annonc/mdz398 · Cited by 263
Suls, J., Mogavero, J. N., Falzon, L., Pescatello, L. S., Hennessy, E. A., & Davidson, K. W. (2020) · Health Psychology Review · doi:10.1080/17437199.2019.1691622 · Cited by 120
Galante, J., Friedrich, C., Collaboration of Mindfulness Trials (CoMinT), Aeamla-Or, N., Arts-de Jong, M., Barrett, B., Bögels, S. M., Buitelaar, J. K., Checovich, M. M., Christopher, M. S., et al. (2023) · Nature Mental Health · doi:10.1038/s44220-023-00081-5 · Cited by 113
Metadata
- Authors
- Tudur Smith, C., Marcucci, M., Nolan, S. J., Iorio, A., Sudell, M., Riley, R., Rovers, M. M., & Williamson, P. R.
- Venue
- Cochrane Database of Systematic Reviews
- Publisher
- Wiley
- Published
- 2016-09-06
- Volume
- 2016
- Issue
- 9
- Pages
- MR000007-MR000007
- ISSN
- 1361-6137 (ISSN-L)
- 1465-1858
- OpenAlex
- W2052335770
- Cited by
- 183
- References
- 6
- FWCI
- 14.97
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Individual participant data meta-analyses compared with meta-analyses based on aggregate data. https://app.cochat.ai/papers/10.1002/14651858.mr000007.pub3
- The paper
- Tudur Smith, C., Marcucci, M., Nolan, S. J., Iorio, A., Sudell, M., Riley, R., Rovers, M. M., & Williamson, P. R. (2016). Individual participant data meta-analyses compared with meta-analyses based on aggregate data. Cochrane Database of Systematic Reviews. https://doi.org/10.1002/14651858.mr000007.pub3
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.