The relative benefits of meta-analysis conducted with individual participant data versus aggregated data
The DOI matches its Crossref record; 42 of the 43 references it cites were found in a registry or an index.
Abstract
The authors describe the relative benefits of conducting meta-analyses with (a) individual participant data (IPD) gathered from the constituent studies and (b) aggregated data (AD), or the group-level statistics (in particular, effect sizes) that appear in reports of a study's results. Given that both IPD and AD are equally available, meta-analysis of IPD is superior to meta-analysis of AD. IPD meta-analysis permits synthesists to perform subgroup analyses not conducted by the original collectors of the data, to check the data and analyses in the original studies, to add new information to the data sets, and to use different statistical methods. However, the cost of IPD meta-analysis and the lack of available IPD data sets suggest that the best strategy currently available is to use both approaches in a complementary fashion such that the first step in conducting an IPD meta-analysis would be to conduct an AD meta-analysis. Regardless of whether a meta-analysis is conducted with IPD or AD, synthesists must remain vigilant in how they interpret their results. They must avoid ecological fallacies, Simpson's paradox, and interpretation of synthesis-generated evidence as supporting causal inferences.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“While the majority of published meta-analyses compare group means and yield only a single estimate of an intervention effect (Alemayehu, 2011), meta-analyses of the rich individual-level data from single case experiments can provide detailed knowledge of individual patient, therapist, and context variables that influence treatment effectiveness (Cooper & Patall, 2009; Iwakabe & Gazzola, 2009; Th”
From Acceptance and Commitment Therapy modules: Differential impact on treatment processes and outcomes (2016) · cites this paper as Cooper & Patall, 2009
Citations
Cites
Ecological Correlations and the Behavior of Individuals
Robinson, W. S. (1950) · American Sociological Review · doi:10.2307/2087176 · Cited by 4,513
The Handbook of Research Synthesis.
Howel, D., Cooper, H., & Hedges, L. V. (1995) · Biometrics · doi:10.2307/2533295 · Cited by 3,642
Self-citationFixed-and random-effects models in meta-analysis.
Hedges, L. V., & Vevea, J. L. (1998) · Psychological Methods · doi:10.1037/1082-989x.3.4.486 · Cited by 2,977
Publication Bias in Meta‐Analysis: Prevention, Assessment and Adjustments
(2005) · doi:10.1002/0470870168 · Cited by 2,368
The interpretation of interaction in contingency tables
Simpson, E. H. (1951) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1951.tb00088.x · Cited by 2,301 · cited 1×
“Related to the ecological fallacy is another phenomenon called “Simpson’s paradox” (Simpson, 1951).”
BackgroundMeta-analysis of psychotherapy outcome studies.
Smith, M. L., & Glass, G. V. (1977) · American Psychologist · doi:10.1037/0003-066x.32.9.752 · Cited by 2,154
Lipsey, M. W., & Wilson, D. B. (1993) · American Psychologist · doi:10.1037/0003-066x.48.12.1181 · Cited by 2,033 · cited 3×
“Mathew and Nordström (1999) then relaxed some of Olkin and Sampson’s assumptions and demonstrated that the estimate of the treatment main effect also was the same for IPD meta-analysis and AD meta-analysis even when observations were not independent within studies and the observations had different covariation matrice…”
MethodResultResearch Synthesis and Meta-Analysis: A Step-by-Step Approach
Cooper, H. (2017) · doi:10.4135/9781071878644 · Cited by 1,716
Self-citationHow Many Studies Do You Need?: A Primer on Statistical Power for Meta-Analysis
Valentine, J. C., Pigott, T. D., & Rothstein, H. R. (2010) · Journal of Educational and Behavioral Statistics · doi:10.3102/1076998609346961 · Cited by 1,294
Interpersonal expectancy effects: the first 345 studies
Rosenthal, R., & Rubin, D. B. (1978) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00075506 · Cited by 1,158
Cited by
Items 301–339 of 339
Simpson's Paradox in Psychology
Mangalam, M. (2022) · Oxford University Press eBooks · doi:10.1093/obo/9780199828340-0301 · Cited by 1
Hall, K. (2013) · Digital Commons - University of South Florida (University of South Florida)
Bash, K. L. (2015) · Lincoln (University of Nebraska)
An Empirical Assessment of Meta-Analysis Estimates from Multi- level Studies
Misran, N. A., & Idris, N. R. N. (2015)
INDIVIDUAL PATIENT DATA META A NALYSIS OF PROGNOSTIC FACTOR STUDIES
Mohammed, G., & Abo-Zaid, A. (2011)
Temple, J. (2019) · University of Liverpool · doi:10.17638/03033174
Daly, B. P., Herres, J. L., Litke, S. G., Domlyn, A. M., & Weist, M. D. (2020) · Clinical Psychology Science and Practice · doi:10.1111/cpsp.12376
Five Things that Went Wrong with Media Violence Research
Grimes, T., & Dailey, S. L. (2021) · Studies in media and communications · doi:10.1108/s2050-206020210000020015
Efficient Integration of Aggregate Data and Individual Patient Data in One-Way Mixed ModelsPreprint
Agarwala, N., Park, J., & Roy, A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.01157
Онтология «я» как источник проблематизации знания и горизонт познавательных возможностей человека
Борисович, А. И. (2013) · Электронный научный архив ТПУ: Главная страница (Tomsk Polytechnic University)
Schneider, M., Merz, S., Stricker, J., Smedt, B. D., Torbeyns, J., Verschaffel, L., & Luwel, K. (2021) · Journal of Numerical Cognition · doi:10.5964/jnc.7617
“Generally, meta-analyses with individual participants’ data, such as the data used by EEA, are possible and offer many advantages over meta-analyses averaging across effect sizes, as done by SEA (Cooper & Patall, 2009).”
MethodLevesque, R. J. R. (2018) · Springer eBooks · doi:10.1007/978-3-319-33228-4_568
Levesque, R. J. R. (2011) · Springer eBooks · doi:10.1007/978-1-4419-1695-2_568
Meta-Analysis within a Single Paper Using Individual-Participant Data: A Simpler Alternative
Kwong, J. Y. Y., & Wong, K. F. E. (2022) · Academy of Management Proceedings · doi:10.5465/ambpp.2022.10395abstract
Wang, Q. (2022) · MOspace Institutional Repository (University of Missouri) · doi:10.32469/10355/91695 · Cited by 1
Weisel, K. K., Baumeister, H., & Ebert, D. D. (2023) · Springer eBooks · doi:10.1007/978-3-662-65816-1_24
Prediction of genes encoding toxin of Mycobacterium tuberculosis H37RV strainPreprint
Genemo, T. G., & Dinka, H. (2025) · Research Square · doi:10.21203/rs.3.rs-5966480/v1
Oakes, H., Eibach, R. P., & Bergsieker, H. B. (2025) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2025.104736
Palantza, C., Morgan, K., Welton, N. J., Micklitz, H. M., Sander, L., & Feder, G. S. (2025) · PLoS ONE · doi:10.1371/journal.pone.0306669 · Cited by 2
“…of aggregate data are limited to the outcomes reported in publications and can only explore study-level moderators which often lack statistical power and are vulnerable to ecological bias, where covariate effects at the study-level do not accord with those seen at the individual level [28].”
Barnhofer, T., Niemi, M., Michalak, J., Velana, M., Williams, J. M., Chiesa, A., Eisendrath, S. J., Delucchi, K., Segal, Z. V., Cladder-Micus, M. B., et al. (2025) · medRxiv · doi:10.1101/2025.06.09.25329249
Psychological distress and compliance with sanitary measures during the Covid-19 pandemic
Hecker, I., Wallez, S., Scarlett, H., Ayuso‐Mateos, J. L., Bryant, R. A., Caggiu, G., Conflitti, C., Gémes, K., Haro, J. M., Lorant, V., et al. (2025) · PLoS ONE · doi:10.1371/journal.pone.0317272 · Cited by 3
“IPD meta-analyses are recognized for providing a more comprehensive assessment of pooled data when compared to aggregate data analyses [68].”
Vossoughi, N., Munoz-Salgado, A., Aleibar, D., Meadows, L. A., & Sekaquaptewa, D. (2025) · Social Psychology of Education · doi:10.1007/s11218-025-10129-2
“As mentioned above, we present the results for Studies 2a and 2b using a mega-anal - ysis (Cooper & Patall, 2009; Curran & Hussong, 2009), in which we combined both datasets into a single large dataset for analysis while including Study ID (2a or 2b) as a factor.”
Brunner, M., Preckel, F., Goetz, T., Lüdtke, O., & Keller, L. (2025) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/6pmhr_v2
Marcucci, M., Legrand, M. M., & Devereaux, P. J. (2025) · European Heart Journal · doi:10.1093/eurheartj/ehaf691
Chick, G., Dong, E., Burns, R. C., Yeh, C.-K., Moreira, J. C., & Sidik, K. (2025) · Cross-Cultural Research · doi:10.1177/10693971251395522
“Because they employ original raw data rather than summary statistics, IPD meta-analyses are generally preferred over AD meta-analyses (Cooper & Patall, 2009).”
Shimshock, C. J., Sisson, N. M., Chee, P. X., Stellar, J. E., Impett, E. A., & Le, B. M. (2025) · Personality and Social Psychology Bulletin · doi:10.1177/01461672251389045
“We chose meta-analysis as opposed to other analytic techniques (e.g., integrative data analysis) given that it accommodates variation in measurement across studies (Cooper & Patall, 2009).”
Do We Regret Group Actions More Than Group Inactions?
Wang, X., van Zomeren, M., & Epstude, K. (2025) · Social Psychology · doi:10.1027/1864-9335/a000586
Perceptions of an ally who confronts racism: The role of displayed emotion and response type
Lopez, A., & Dickter, C. L. (2026) · Group Processes & Intergroup Relations · doi:10.1177/13684302261422160
“To integrate the data across studies in order to increase statistical power and generalizability, we conducted an integrative meta-analysis, which pools the raw data from multiple studies into a large dataset (Cooper & Patall, 2009; Curran & Hussong, 2009).”
Chow, J. C., & Martinez, R. G. (2026) · Educational Psychology Review · doi:10.1007/s10648-026-10152-5
“…meta-analysts consider individual-data approaches as a way, though cumbersome and complex, to counter the ecological fallacy within meta-analysis (Cooper & Patall, 2009; Kaufmann et al., 2016); CAPERS may provide an aligned, but methodologically distinct response to the ecological fallacy by…”
Kozyreva, A., Lorenz-Spreen, P., Herzog, S., Ecker, U., Lewandowsky, S., Hertwig, R., Ali, A., Bak-Coleman, J., Barzilai, S., Basol, M., et al. (2024) · MPG.PuRe (Max Planck Society)
Beck, E. D., Willroth, E. C., Delius, J. A. M., Bennett, D. A., Barnes, L. L., James, B. D., Lipton, R. B., Katz, M., Hassing, L. B., Huisman, M. A., et al. (2026) · Personality and Social Psychology Bulletin · doi:10.1177/01461672261451901 · Cited by 2
A Meta-Analysis on Application of Protection Motivation Theory Toward Green Purchase Behaviour
Le, N., & Huynh, T. N. (2026) · Springer proceedings in business and economics · doi:10.1007/978-981-95-9113-8_1
Bozionelos, N., Mukhuty, S., Singh, S. K., & Bozionelos, G. (2026) · Personnel Review · doi:10.1108/pr-12-2025-1427
Educational inequality across countries: An integrative data analysis of TIMSS 2023 data
Patsis, P., Campos, D., & Scherer, R. (2026) · Studies In Educational Evaluation · doi:10.1016/j.stueduc.2026.101683
Aikawa, S. M., Pietri, E. S., Brown, E. J., & Parks, K. (2026) · Behavioral Sciences · doi:10.3390/bs16101776
“To explore the consistency of the effects of the flag condition and the Participant Race by Flag Condition interaction across studies, we conducted an internal mega-analysis (e.g., Cooper & Patall, 2009; Curran & Hussong, 2009).”
Hien, D. A., Blakey, S. M., Norman, S. B., Killeen, T. K., Saavedra, L. M., Back, S. E., López‐Castro, T., Simpson, T. L., Swarbrick, M., Petrakis, I. L., et al. (2026) · European Journal of Psychotraumatology · doi:10.1080/20008066.2026.2715184
“Through greater power and measurement precision, Project Harmony detected effects that likely could never have emerged using conventional meta-analytic methods (Cooper & Patall, 2009; Lubinski & Humphreys, 1996).”
Metadata
- Authors
- Cooper, H., & Patall, E. A.
- Affiliations
- Duke University (US)
- Venue
- Psychological Methods
- Published
- 2009
- Volume
- 14
- Issue
- 2
- Pages
- 165-176
- ISSN
- 1082-989X (ISSN-L)
- 1939-1463
- OpenAlex
- W2003323761
- Semantic Scholar
- ea908786103694ec9857ffd3dd31530573436cdc
- Cited by
- 339
- References
- 43
- Citations by year
- 2026: 10 · 2025: 25 · 2024: 18 · 2023: 21 · 2022: 30 · 2021: 22 · 2020: 22 · 2019: 20 · 2018: 24 · 2017: 16 · 2016: 26 · 2015: 32 · 2014: 17 · 2013: 16 · 2012: 14
- FWCI
- 16.13
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Psychological Methods (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Keywords
- individual patient data, aggregated data, meta-analysis, subgroup analysis, ecological fallacy, Simpson's paradox, causal inference, effect size, data availability
- MeSH terms
- Data Collection, Data Interpretation, Statistical, Humans, Meta-Analysis as Topic, Models, Statistical
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.