The relative benefits of meta-analysis conducted with individual participant data versus aggregated data
The DOI matches its Crossref record; 42 of the 43 references it cites were found in a registry or an index.
Abstract
The authors describe the relative benefits of conducting meta-analyses with (a) individual participant data (IPD) gathered from the constituent studies and (b) aggregated data (AD), or the group-level statistics (in particular, effect sizes) that appear in reports of a study's results. Given that both IPD and AD are equally available, meta-analysis of IPD is superior to meta-analysis of AD. IPD meta-analysis permits synthesists to perform subgroup analyses not conducted by the original collectors of the data, to check the data and analyses in the original studies, to add new information to the data sets, and to use different statistical methods. However, the cost of IPD meta-analysis and the lack of available IPD data sets suggest that the best strategy currently available is to use both approaches in a complementary fashion such that the first step in conducting an IPD meta-analysis would be to conduct an AD meta-analysis. Regardless of whether a meta-analysis is conducted with IPD or AD, synthesists must remain vigilant in how they interpret their results. They must avoid ecological fallacies, Simpson's paradox, and interpretation of synthesis-generated evidence as supporting causal inferences.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“While the majority of published meta-analyses compare group means and yield only a single estimate of an intervention effect (Alemayehu, 2011), meta-analyses of the rich individual-level data from single case experiments can provide detailed knowledge of individual patient, therapist, and context variables that influence treatment effectiveness (Cooper & Patall, 2009; Iwakabe & Gazzola, 2009; Th”
From Acceptance and Commitment Therapy modules: Differential impact on treatment processes and outcomes (2016) · cites this paper as Cooper & Patall, 2009
Citations
Cites
Ecological Correlations and the Behavior of Individuals
Robinson, W. S. (1950) · American Sociological Review · doi:10.2307/2087176 · Cited by 4,513
The Handbook of Research Synthesis.
Howel, D., Cooper, H., & Hedges, L. V. (1995) · Biometrics · doi:10.2307/2533295 · Cited by 3,642
Self-citationFixed-and random-effects models in meta-analysis.
Hedges, L. V., & Vevea, J. L. (1998) · Psychological Methods · doi:10.1037/1082-989x.3.4.486 · Cited by 2,977
Publication Bias in Meta‐Analysis: Prevention, Assessment and Adjustments
(2005) · doi:10.1002/0470870168 · Cited by 2,368
The interpretation of interaction in contingency tables
Simpson, E. H. (1951) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1951.tb00088.x · Cited by 2,301 · cited 1×
“Related to the ecological fallacy is another phenomenon called “Simpson’s paradox” (Simpson, 1951).”
BackgroundMeta-analysis of psychotherapy outcome studies.
Smith, M. L., & Glass, G. V. (1977) · American Psychologist · doi:10.1037/0003-066x.32.9.752 · Cited by 2,154
Lipsey, M. W., & Wilson, D. B. (1993) · American Psychologist · doi:10.1037/0003-066x.48.12.1181 · Cited by 2,033 · cited 3×
“Mathew and Nordström (1999) then relaxed some of Olkin and Sampson’s assumptions and demonstrated that the estimate of the treatment main effect also was the same for IPD meta-analysis and AD meta-analysis even when observations were not independent within studies and the observations had different covariation matrice…”
MethodResultResearch Synthesis and Meta-Analysis: A Step-by-Step Approach
Cooper, H. (2017) · doi:10.4135/9781071878644 · Cited by 1,716
Self-citationHow Many Studies Do You Need?: A Primer on Statistical Power for Meta-Analysis
Valentine, J. C., Pigott, T. D., & Rothstein, H. R. (2010) · Journal of Educational and Behavioral Statistics · doi:10.3102/1076998609346961 · Cited by 1,294
Interpersonal expectancy effects: the first 345 studies
Rosenthal, R., & Rubin, D. B. (1978) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00075506 · Cited by 1,158
Cited by
Items 101–200 of 339
Polanin, J. R. (2017) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2017.12.014 · Cited by 35
Systematic reviews and meta-analysis in nutrition research
Kelley, G. A., & Kelley, K. S. (2019) · British Journal Of Nutrition · doi:10.1017/s0007114519002241 · Cited by 38
Ho, T. J. H., Lee, C. C. S., Wong, S. N., & Lau, Y. (2018) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2018.09.019 · Cited by 38
Incidental treatment effects of CBT on suicidal ideation and hopelessness
Handley, T. E., KAY‐LAMBKIN, F. J., Baker, A., Lewin, T. J., Kelly, B. J., Inder, K., Attia, J. R., & Kavanagh, D. J. (2013) · Journal of Affective Disorders · doi:10.1016/j.jad.2013.06.005 · Cited by 37
“Integrating data across independent studies has become increasingly common over the past decade, and has been associated with a range of methodological benefits (Stewart et al., 2012; Cooper and Patall, 2009; Stewart and Tierney, 2002).”
BackgroundMantua, J., Bessey, A. F., Mickelson, C. A., Choynowski, J. J., Noble, J. J., Burke, T. M., McKeon, A. B., & Sowden, W. J. (2021) · SLEEP · doi:10.1093/sleep/zsaa221 · Cited by 33
Kelley, G. A., & Kelley, K. S. (2014) · BMC Musculoskeletal Disorders · doi:10.1186/1471-2474-15-121 · Cited by 42
“from investigators as well as the increased costs and time associated with the conduct of such [48].”
BackgroundOvercoming obstacles in obtaining individual participant data for meta‐analysis
Polanin, J. R., & Williams, R. T. (2016) · Research Synthesis Methods · doi:10.1002/jrsm.1208 · Cited by 36
“A new academic journal, Archives of Scientific Psychology, announced that in order to publish a manuscript in the Archives, authors must submit their data to an international repository of digital social science data (Cooper & VandenBos, 2013).”
BackgroundTrudgill, D. I. N., Gorey, K. M., & Donnelly, E. A. (2020) · Humanities and Social Sciences Communications · doi:10.1057/s41599-020-00584-x · Cited by 42
“…were all discrete, they were combined into a common 2 HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | (2020) 7:89 | https://doi.org/10.1057/s41599-020-00584-x database, allowing for a powerful overall, sample-weighted pointestimate along with its 95% CI (Cooper, 2017; Cooper and Patall, 2009).”
InfluentialMichie, S., Johnson, B. T., & Johnston, M. (2015) · Health Psychology Review · doi:10.1080/17437199.2014.912538 · Cited by 33
Loh, Y. L., Yaw, Q. P., & Lau, Y. (2023) · International Journal of Obesity · doi:10.1038/s41366-023-01304-6 · Cited by 42
“Fifthly, meta-analysis results could be subjected to ecological fallacy or Simpson’s paradox [114].”
Meta-analyses in sales research
Johnson, J. S., & Jaramillo, F. (2017) · Journal of Personal Selling and Sales Management · doi:10.1080/08853134.2017.1286596 · Cited by 37
“In addition, Cooper and Patall (2009) also distinguished between aggregated data (AD) and individual participant data (IPD) meta-analyses.”
BackgroundKcomt, L., & Gorey, K. M. (2017) · Journal of Social Work in End-of-Life & Palliative Care · doi:10.1080/15524256.2017.1387214 · Cited by 41
“This is the most desirable method of combining the findings of independent studies allowing for the most powerful overall estimates (Cooper, 2017; Cooper & Patall, 2009).”
MethodKaufmann, E., Reips, U.-D., & Maag Merki, K. (2016) · Zeitschrift für Psychologie · doi:10.1027/2151-2604/a000251 · Cited by 31
“As mentioned by Cooper and Patall (2009), IPD meta-analysis requires many more staff for data collection, entry, and cleaning than classical meta-analysis.”
BackgroundSaavedra, L. M., Morgan-López, A. A., Hien, D. A., López-Castro, T., Ruglass, L. M., Back, S. E., Fitzpatrick, S., Norman, S. B., Killeen, T. K., Ebrahimi, C. T., et al. (2021) · Contemporary Clinical Trials · doi:10.1016/j.cct.2021.106479 · Cited by 31
Beck, E. D., Yoneda, T. B., James, B. D., Bennett, D. A., Hassenstab, J. J., Katz, M. J., Lipton, R. B., Morris, J. C., Mroczek, D. K., & Graham, E. K. (2023) · Alzheimer s & Dementia · doi:10.1002/alz.13523 · Cited by 45
Technology effectiveness in the mathematics classroom: a systematic review of meta-analytic research
Young, J., Gorumek, F., & Hamilton, C. (2018) · Journal of Computers in Education · doi:10.1007/s40692-018-0104-2 · Cited by 38
“Moderators are also important because they identify statistical interactions, which do not imply causation but rather add context to effect size results (Cooper and Patall 2009).”
BackgroundFrancis, Z. L., Milyavskaya, M., Lin, H., & Inzlicht, M. (2018) · Social Psychology · doi:10.1027/1864-9335/a000348 · Cited by 41
“Meta-analysis using raw data, instead of using summary statistics, allows for increased power and ability to examine moderators and is preferred when the raw data is available (Cooper & Patall, 2009; Curran & Hussong, 2009).”
MethodCheng, L. J., Kumar, P. A., Wong, S. N., & Lau, Y. (2019) · AIDS and Behavior · doi:10.1007/s10461-019-02691-6 · Cited by 30
Wolff, F., & Møller, J. (2022) · Educational Psychology Review · doi:10.1007/s10648-022-09686-1 · Cited by 44
“…of meta-analyses” (Kaufmann et al., 2016, p. 157)—has at least three central advantages compared to classic aggregated person data (APD) meta-analyses, which synthesize data from 1 3 studies using the summary statistics presented in the respective publications (e.g., Cooper & Patall, 2009).”
MethodA family of experiments on test-driven development
Santos, A., Vegas, S., Dieste, O., Uyaguari, F., Tosun, A., Fucci, D., Turhan, B., Scanniello, G., Romano, S., Karac, I., et al. (2021) · Empirical Software Engineering · doi:10.1007/s10664-020-09895-8 · Cited by 30
Allen, J., Inder, K., Lewin, T. J., Attia, J. R., KAY‐LAMBKIN, F. J., Baker, A., Hazell, T., & Kelly, B. J. (2013) · BMC Medical Research Methodology · doi:10.1186/1471-2288-13-122 · Cited by 34
“Still, even this issue may be addressed simply by making data sets available to researchers only under the same rules of confidentiality that applied when the data was first collected” [1].”
BackgroundMethodInfluentialHe, J., Cai, Z., & Fan, X. (2017) · Obesity Research & Clinical Practice · doi:10.1016/j.orcp.2017.03.004 · Cited by 32
Bonafiglia, J. T., Swinton, P. A., Ross, R., Johannsen, N. M., Martin, C. K., Church, T. S., Slentz, C. A., Ross, L. M., Kraus, W. E., Walsh, J. J., et al. (2022) · Sports Medicine · doi:10.1007/s40279-022-01725-9 · Cited by 29
“Compared with aggregate data meta-analyses, IPD meta-analyses permit the ability to investigate potential moderators, provide more precise estimates, and enable greater flexibility in statistical modelling by unrestricting assumptions of the distribution of underlying change scores [16].”
ResultKern, M., Trumpold, K., & Zapf, D. (2021) · European Journal of Work and Organizational Psychology · doi:10.1080/1359432x.2021.1873771 · Cited by 35
“Errors and measures can be checked and the analyses are standardized, thus maximizing the statistical power (Cooper & Patall, 2009).”
BackgroundChildhood Obesity Research Demonstration Project: Cross-Site Evaluation Methods
O’Connor, D. P., Lee, R. E., Mehta, P. D., Thompson, D. I., Bhargava, A., Carlson, C. D., Kao, D., Layne, C. S., Ledoux, T. A., O’Connor, T. M., et al. (2015) · Childhood Obesity · doi:10.1089/chi.2014.0061 · Cited by 28
“, 24, 25 Including the intervention components and As an example, the first row in Table 1 represents the implementation of the full CORD model with integrated PH and PC Plus interventions.”
Research Synthesis and Meta-Analysis of Single-Case Designs
Pustejovsky, J., & Ferron, J. (2017) · doi:10.4324/9781315517698.ch12 · Cited by 76
“Although individual-participant data has several advantages for meta-analysis of between-groups designs (Cooper & Patall, 2009), its use is still relatively uncommon there because raw data are not always readily available.”
BackgroundCustomer-Related Social Stressors: Meaning and Consequences Across Service Jobs
Dudenhöffer, S., & Dormann, C. (2015) · Journal of Personnel Psychology · doi:10.1027/1866-5888/a000132 · Cited by 43
Weisel, K. K., Lehr, D., Heber, E., Zarski, A.-C., Berking, M., Riper, H., & Ebert, D. D. (2018) · Journal of Medical Internet Research · doi:10.2196/jmir.9387 · Cited by 28
“To overcome this issue, data can be pooled from single studies by adding individual participant data in one dataset for common analyses [16].”
MethodHo, T. J. H., Cheng, L. J., & Lau, Y. (2021) · Public Health Nutrition · doi:10.1017/s1368980021001117 · Cited by 41
“Third, the results of this meta-analysis are subject to ecological fallacy and/or Simpson ’ s paradox as with any meta-analysis (70) .”
Meta-Analysis of Research in Social and Personality Psychology
Johnson, B. T., & Eagly, A. H. (2014) · Cambridge University Press eBooks · doi:10.1017/cbo9780511996481.031 · Cited by 28
Combining individual participant and aggregated data in a meta‐analysis with correlational studies
Pigott, T., Williams, R. T., & Polanin, J. R. (2012) · Research Synthesis Methods · doi:10.1002/jrsm.1051 · Cited by 27
“Although access to more data sets will no doubt encourage reviewers to include individual data into metaanalyses, there are a number of other difficulties faced by reviewers who combine IPD and AD in a single metaanalysis (Cooper and Patall, 2009).”
BackgroundTesting moderation in network meta‐analysis with individual participant data
Dagne, G. A., Brown, C. H., Howe, G. W., Kellam, S. G., & Liu, L. (2016) · Statistics in Medicine · doi:10.1002/sim.6883 · Cited by 28
“[44] to describe the condition where direct and indirect comparisons of two interventions do not differ.”
The daily association between affect and alcohol use: A meta-analysis of individual participant dataPreprint
Dora, J., Piccirillo, M., Foster, K. T., Arbeau, K. J., Armeli, S., Auriacombe, M., Bartholow, B., Beltz, A., Blumenstock, S. M., Bold, K., et al. (2022) · Psychological bulletin · doi:10.31234/osf.io/xevct · Cited by 117
“Such an individual participant data meta-analysis is superior for several additional reasons (H. Cooper & Patall, 2009).”
BackgroundMethodological Considerations for the Study of Adult Development and Aging
MacDonald, S. W. S., & Stawski, R. S. (2015) · Elsevier eBooks · doi:10.1016/b978-0-12-411469-2.00002-9 · Cited by 30
Harte, N., Aaron, R. V., Bhattiprolu, K., Bisby, M. A., Gandy, M., Hathway, T., Dear, B. F., & Dudeney, J. E. (2024) · Journal of Psychosomatic Research · doi:10.1016/j.jpsychores.2024.111945 · Cited by 38
Ebert, D. D., Buntrock, C., Reins, J. A., Zimmermann, J., & Cuijpers, P. J. (2018) · BMJ Open · doi:10.1136/bmjopen-2017-018582 · Cited by 27
“By pooling the primary data of individual trials, it is possible to conduct analyses not reported in original studies and obtain large enough sample sizes with sufficient power to examine the effects in relevant subgroups and identify outcome moderators.(30) The present study aims to examine the short-term and long-te…”
Evaluating Model-Driven Development Claims with Respect to Quality: A Family of Experiments
Panach, J. I., Dieste, Ó., Marín, B., España, S., Vegas, S., Pastor, Ó., & Juristo, N. (2018) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2018.2884706 · Cited by 27
“the raw data and it is a powerful technique for identifying moderator variables [10].”
MethodDeclercq, L., Jamshidi, L., Fernández‐Castilla, B., Moeyaert, M., Beretvas, S. N., Ferron, J. M., & Noortgate, W. V. D. (2020) · Multivariate Behavioral Research · doi:10.1080/00273171.2020.1822148 · Cited by 33
“…the digital revolution has enabled sharing of (individual participant) data on a larger scale, the AD metaanalytic approach has been scrutinized and consensus came to be that IPD meta-analysis is to be preferred instead (Cooper & Patall, 2009; Stewart & Tierney, 2002; Tudur Smith et al., 2016).”
BackgroundMethodInfluentialLiebst, L. S., Ejbye‐Ernst, P., de Bruin, M., Thomas, J., & Lindegaard, M. R. (2022) · Scientific Reports · doi:10.1038/s41598-022-05270-3 · Cited by 28
Jewish Population in the United States, 2015
Sheskin, I. M., & Dashefsky, A. (2016) · American Jewish year book · doi:10.1007/978-3-319-24505-8_5 · Cited by 26
Data Integration Approaches to Longitudinal Growth Modeling
Marcoulides, K. M., & Grimm, K. J. (2016) · Educational and Psychological Measurement · doi:10.1177/0013164416664117 · Cited by 26
“…data fusion, longitudinal growth modeling Research synthesis involves collecting, combining, and summarizing research for a specific study question (Cooper & Patall, 2009) in an attempt to come to an 1Arizona State University, Tempe, AZ, USA Corresponding Author: Katerina M. Marcoulides,…”
BackgroundMethodInfluentialNg, M. Y., DiVasto, K. A., Cootner, S., Gonzalez, N.-A.-R., & Weisz, J. R. (2023) · Clinical Psychology: Science and Practice · doi:10.1111/cpsp.12367 · Cited by 25
“Perhaps our most striking finding is that completed RCTs are a substantial source of untested CMs. Individual participant data (IPD) meta-analysis (Cooper & Patall, 2009) and metaanalytic structural equation modeling (MASEM; Cheung & Hafdahl, 2016) can quantify mean conjoint pathways from…”
BackgroundEngeda, J. C., Stackhouse, A., White, M., Rosamond, W. D., Lhachimi, S. K., Lund, J. L., Keyserling, T. C., & Avery, C. L. (2019) · Diabetes Research and Clinical Practice · doi:10.1016/j.diabres.2019.04.005 · Cited by 33
Kim, M., & Cappella, J. N. (2019) · Journal of Health Communication · doi:10.1080/10810730.2019.1668090 · Cited by 29
“Another metaanalytic approach using individual participant data rather than aggregate data (Cooper & Patall, 2009) was also employed.”
MethodCheng, L. J., Jiang, Y., Wu, X. V., & Wang, W. (2019) · Journal of Advanced Nursing · doi:10.1111/jan.14255 · Cited by 32
Deciphering ‘Urge to Buy’: A Meta-Analysis of Antecedents
Paul, J., Kaur, J., Arora, S., & Singh, S. V. (2022) · International Journal of Market Research · doi:10.1177/14707853221106317 · Cited by 28
“In the process, the findings are aggregated to establish the effective strength and generalizability of their causal links (Cooper & Patall, 2009; Pan & Sparks, 2012).”
BackgroundPaths to Positive Development: a Model of Outcomes in the New Zealand Youth Transitions Study
Boden, J. M., Sanders, J., Munford, R., Liebenberg, L., & McLeod, G. F. H. (2015) · Child Indicators Research · doi:10.1007/s12187-015-9341-3 · Cited by 31
Meta‐analysis in Stata using gllamm
Bagos, P. G. (2015) · Research Synthesis Methods · doi:10.1002/jrsm.1157 · Cited by 30
“…termed individual patient data (IPD) methods, are superior in many circumstances compared with the usual summary-data methods, because they avoid the normality assumptions and they do not need continuity correction in case of zero cell counts (Cooper and Patall, 2009; Stewart and Tierney, 2002).”
Meta‐analysis of Studies with Multiple Contrasts and Differences in Measurement Scales
McShane, B. B., & Böckenholt, U. (2021) · Journal of Consumer Psychology · doi:10.1002/jcpy.1236 · Cited by 27
van Loo, H. M., van den Heuvel, E. R., Schoevers, R. A., Anselmino, M., Carney, R. M., Denollet, J., Doyle, F., Freedland, K. E., Grace, S. L., Hosseini, S. H., et al. (2014) · BMC Medicine · doi:10.1186/s12916-014-0242-y · Cited by 28
“This was done to account for systematic differences between studies and thus the expectation that different effect sizes could be underlying the different studies [21,22].”
Unpacking TPACK in Mathematics Education Research: A Systematic Review of Meta-Analyses
Young, J. (2016) · International Journal of Educational Methodology · doi:10.12973/ijem.2.1.19 · Cited by 30
“Moderators are important because they cause statistical interactions, but associations do not mean causation (Cooper & Patall, 2009; Russell & Gilliland, 1995).”
BackgroundVeroniki, A. A., Straus, S. E., Ashoor, H. M., Hamid, J. S., Hemmelgarn, B. R., Holroyd-Leduc, J. M., Majumdar, S. R., McAuley, G., & Tricco, A. C. (2016) · BMJ Open · doi:10.1136/bmjopen-2015-010251 · Cited by 23
Maagaard, M., Andersen, J. H., Jæger, P., & Mathiesen, O. (2024) · Regional Anesthesia & Pain Medicine · doi:10.1136/rapm-2023-105098 · Cited by 26
“32 We supplemented the meta-analyses with PIs.”
Ellis, A. G., Reginster, J., Luo, X., Bushmakin, A. G., Williams, R., Sutradhar, S., Mirkin, S., & Jansen, J. P. (2014) · Current Medical Research and Opinion · doi:10.1185/03007995.2014.908279 · Cited by 24
Sobel, M. E., Madigan, D., & Wang, W. (2016) · Psychometrika · doi:10.1007/s11336-016-9507-z · Cited by 26
“Individual participant data (IPD) meta-analyses, of primary focus herein, offer numerous advantages (Cooper & Patall, 2009; Simmonds et al., 2005), and are becoming increasingly common (as evidenced, for example, by the many papers listed on the website of the Cochrane Individual Participant…”
BackgroundWitkiewitz, K. A., Hallgren, K. A., O'Sickey, A. J., Roos, C. R., & Maisto, S. A. (2015) · Drug and Alcohol Dependence · doi:10.1016/j.drugalcdep.2015.11.001 · Cited by 21
A Procedure and Guidelines for Analyzing Groups of Software Engineering Replications
Santos, A., Vegas, S., Oivo, M., & Juristo, N. (2019) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2019.2935720 · Cited by 31
Parr, N. J., Schweer‐Collins, M. L., Darlington, T. M., & Tanner‐Smith, E. E. (2019) · Journal of Adolescence · doi:10.1016/j.adolescence.2019.10.009 · Cited by 26
Boden, J. M., Sanders, J., Munford, R., & Liebenberg, L. (2016) · Child Indicators Research · doi:10.1007/s12187-016-9422-y · Cited by 23
Mun, E., Li, X., Lineberry, S., Tan, Z., Huh, D., Walters, S. T., Zhou, Z., & Larimer, M. E. (2020) · Alcohol and Alcoholism · doi:10.1093/alcalc/agaa146 · Cited by 23
Denominational Variations Across American Jewish Communities
Sheskin, I. M., & Hartman, H. J. (2015) · Journal for the Scientific Study of Religion · doi:10.1111/jssr.12189 · Cited by 20
Veroniki, A. A., Soobiah, C., Tricco, A. C., Elliott, M. J., & Straus, S. E. (2015) · BMJ Open · doi:10.1136/bmjopen-2014-007103 · Cited by 20
Elnakouri, A., Hubley, C., & McGregor, I. (2021) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2021.104227 · Cited by 22
Lydecker, J. A., Gueorguieva, R. V., Masheb, R. M., White, M. A., & Grilo, C. M. (2019) · International Journal of Eating Disorders · doi:10.1002/eat.23167 · Cited by 21
“Mixed models have well-established advantages for this type off study, particularly the application of consistent data analysis techniques across the studies (Cooper & Patall, 2009; Curran & Hussong, 2009).”
MethodMarcó-García, S., Ariyo, K., Owen, G. S., & David, A. S. (2024) · Psychological Medicine · doi:10.1017/s0033291724000242 · Cited by 18
“Future studies could use an individual patient data meta-analysis (Cooper & Patall, 2009; Riley, Lambert, & Abo-Zaid, 2010) to explore how these effects vary between subgroups and subscales.”
Utilizing Moderated Non-linear Factor Analysis Models for Integrative Data Analysis: A Tutorial
Kush, J. M., Masyn, K. E., Amin‐Esmaeili, M., Susukida, R., Wilcox, H. C., & Musci, R. J. (2022) · Structural Equation Modeling A Multidisciplinary Journal · doi:10.1080/10705511.2022.2070753 · Cited by 23
“Notably, however, Cooper and Patall (2009) differentiate between meta-analysis based on aggregated data, as described above, vs. meta-analysis based on individual participant-level data.”
BackgroundMethodInfluentialIs renoprotection with RAAS blockade a failed paradigm? Have we learnt any lessons so far?
Onuigbo, M. A. C. (2010) · International Journal of Clinical Practice · doi:10.1111/j.1742-1241.2010.02448.x · Cited by 17
Relative efficiency of using summary versus individual data in random‐effects meta‐analysis
Chen, D., Liu, D., Min, X., & Zhang, H. (2020) · Biometrics · doi:10.1111/biom.13238 · Cited by 22
“Another crucial benefit is that the IPD analysis can avoid the so-called “ecological fallacy” (Reade et al., 2008; Cooper and Patall, 2009).”
BackgroundVeroniki, A. A., Straus, S. E., Ashoor, H. M., Stewart, L. A., Clarke, M. J., & Tricco, A. C. (2016) · Trials · doi:10.1186/s13063-016-1238-z · Cited by 16
“ies, and hence results might be prone to several biases [14, 45, 46].”
BackgroundCollectively constructing gender-inclusive work cultures in STEM
Hall, W. M., Schmader, T., Cyr, E. N., & Bergsieker, H. B. (2022) · European Review of Social Psychology · doi:10.1080/10463283.2022.2109294 · Cited by 24
Distribution and Epidemiological Characteristics of Published Individual Patient Data Meta-Analyses
Huang, Y., Mao, C., Yuan, J.-Q., Yang, Z., Di, M., Tam, W., & Tang, J.-L. (2014) · PLoS ONE · doi:10.1371/journal.pone.0100151 · Cited by 16
“IPDMA is generally believed to have advantages over ADMA because IPDMA uses consistent inclusion and exclusion criteria among IPD, thus increasing data sensitivity and specificity with detailed data analysis [2,9,10].”
MethodUsing the SOCIO Chatbot for UML Modelling: A Family of Experiments
Ren, R., Castro, J. W., Santos, A., Dieste, Ó., & Acuña, S. T. (2022) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2022.3150720 · Cited by 22
“Families provide certain advantages for evaluating the effectiveness of SE treatments [14], [15], [16], [17], [18]: (i) researchers can apply consistent pre-processing and analysis techniques (as researchers have access to the raw data) to analyse the experiments and, in turn, increase the…”
BackgroundYou, S. C., Lee, S., Choi, B., & Park, R. W. (2022) · Korean Circulation Journal · doi:10.4070/kcj.2022.0294 · Cited by 21
Callina, K. S., Johnson, S. K., Tirrell, J. M., Batanova, M., Weiner, M. B., & Lerner, R. M. (2017) · Journal of Youth and Adolescence · doi:10.1007/s10964-017-0660-1 · Cited by 23
“In meta-analysis, parameter estimates are pooled from existing studies (Cooper and Patall 2009; Cumming 2013).”
MethodLiebst, L. S., Ejbye‐Ernst, P., de Bruin, M., Thomas, J., & Lindegaard, M. R. (2021) · Transboundary and Emerging Diseases · doi:10.1111/tbed.14094 · Cited by 16
“A prospect of the two current data sets is that they may be pooled into one large and high- powered dataset (Cooper & Patall, 2009).”
BackgroundWhy We Need Systematic Reviews and Meta-Analyses in the Testing and Assessment Literature
Iliescu, D., Rusu, A., Greiff, S., Fokkema, M., & Scherer, R. (2022) · European Journal of Psychological Assessment · doi:10.1027/1015-5759/a000705 · Cited by 18
Research synthesis methods and library and information science: Shared problems, limited diffusion
Sheble, L. (2015) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.23499 · Cited by 18
“Researchers emphasize similarities and differences between documentand data-based syntheses (e.g., Cooper & Patall, 2009; Riley, Lambert, & Abo-Zaid, 2010).”
BackgroundA comparison of Bayesian synthesis approaches for studies comparing two means: A tutorial
Du, H., Bradbury, T. N., Lavner, J. A., Meltzer, A. L., McNulty, J. K., Neff, L. A., & Karney, B. R. (2019) · Research Synthesis Methods · doi:10.1002/jrsm.1365 · Cited by 16
“When the original data from individual studies are available, it is straightforward to conduct an integrative data analysis, also referred to as individual participant-level data meta-analysis, pooled analysis, or mega-analysis.(13,14) In an integrative data analysis, all the raw data from different studies are merged…”
BackgroundMethodInfluentialNetwork Meta-Analysis for Single-Case Design Studies: An Illustration
Mendes, A. B., Jamshidi, L., Noortgate, W. V. D., & Fernández‐Castilla, B. (2022) · Evaluation & the Health Professions · doi:10.1177/01632787211067532 · Cited by 14
No evidence for inequity aversion in non-human animals: a meta-analysis of accept/reject paradigms
Ritov, O., Völter, C. J., Raihani, N., & Engelmann, J. M. (2024) · Proceedings of the Royal Society B Biological Sciences · doi:10.1098/rspb.2024.1452 · Cited by 24
Grilo, C. M., Thompson‐Brenner, H., Shingleton, R. M., Thompson, D., & Franko, D. L. (2021) · International Journal of Eating Disorders · doi:10.1002/eat.23601 · Cited by 17
Kemp, J. J., Barker, D., Benito, K., Herren, J. A., & Freeman, J. B. (2020) · Journal of Clinical Child & Adolescent Psychology · doi:10.1080/15374416.2020.1790378 · Cited by 13
“…to AD-MA, IPD-MA approaches are able to utilize the same analytic approach across trials, run analyses for subgroups not reported in the published literature, and address between-trial differences in participant characteristics (Cooper & Patall, 2009; Debray et al., 2015; Smith et al., 2016).”
MethodMorgan‐López, A. A., McDaniel, H. L., Bradshaw, C. P., Saavedra, L. M., Lochman, J. E., Kaihoi, C. A., Powell, N. P., Qu, L., & Yaros, A. (2022) · Contemporary Clinical Trials · doi:10.1016/j.cct.2022.106705 · Cited by 15
Short and long views of integrative data analysis: Comments on contributions to the special issue.
Shrout, P. E. (2009) · Psychological Methods · doi:10.1037/a0015953 · Cited by 14
“The research framework provided by Curran and Hussong (2009) and by Cooper and Patall (2009) and the tools and examples of models included in the articles by Bauer and Hussong (2009) and by McArdle et al. (2009) make it clear that IDA is a methodological approach that has great potential as a…”
BackgroundMethodResultInfluentialWilliams, A. D. N., Hood, K., Bracken, K., & Shorter, G. W. (2023) · Trials · doi:10.1186/s13063-023-07278-0 · Cited by 16
“For all those involved in clinical trials, we ask you to consider how you collect sex, gender and sexuality data in your trials and is your data collection being inclusive? Even if your data cannot be analysed through these groups, advances in data synthesis such as individual patient data meta-analysis, can better un…”
BackgroundCorporal punishment and violent behavior spectrum: a meta-analytic review
Pan, Q., Chen, S.-R., & Qu, Y. (2024) · Frontiers in Psychology · doi:10.3389/fpsyg.2024.1323784 · Cited by 21
Crocetti, E., Lise, F. D., Pagano, M., Bobba, B., Maratia, F., Meeus, W., & Bacaro, V. (2024) · Identity · doi:10.1080/15283488.2024.2394861 · Cited by 21
“…research becomes available and meta-analytic methodologies, such as the Individual Participant Data (IPD) approach (Stewart et al., 2019) pioneered in the medical field (Cooper & Patall, 2009), become more common also in the social sciences, it would be possible to delve into moderating effects.”
Using robust standard errors to combine multiple regression estimates with meta-analysis
Williams, R. T. (2012) · Loyola eCommons (Loyola University of Chicago) · Cited by 23
Moderator effects differ on alternative effect-size measures
Smithson, M., & Shou, Y. (2016) · Behavior Research Methods · doi:10.3758/s13428-016-0735-z · Cited by 13
The Potassium Cycle and Its Relationship to Recommendation Development
Brouder, S. M., Volenec, J. J., & Murrell, T. S. (2020) · Springer eBooks · doi:10.1007/978-3-030-59197-7_1 · Cited by 20
Godolphin, P. J., Bath, P. M., & Montgomery, A. A. (2019) · BMJ Open · doi:10.1136/bmjopen-2018-025273 · Cited by 16
Enriching Meta-Analytic Models of Summary Data: A Thought Experiment and Case Study
McShane, B. B., & Böckenholt, U. (2019) · Advances in Methods and Practices in Psychological Science · doi:10.1177/2515245919884304 · Cited by 14
“…although metaanalysis is often equated to the analysis of summary data, the analysis of the individual-level data from the studies via a hierarchical model is considered the “gold standard” for meta-analysis (Cooper & Patall, 2009; Haidich, 2010; Simmonds et al., 2005; Stewart & Tierney, 2002).”
MethodLiebst, L. S., Ejbye‐Ernst, P., de Bruin, M., Thomas, J., & Lindegaard, M. R. (2021) · Research Square · doi:10.31234/osf.io/ep8jg · Cited by 13
Dewidar, O., Riddle, A. Y., Ghogomu, E. T., Hossain, A., Arora, P., Bhutta, Z. A., Black, R. E., Cousens, S., Gaffey, M. F., Mathew, C. M., et al. (2021) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2021.05.007 · Cited by 13
MACH14: A Multi-Site Collaboration on ART Adherence Among 14 Institutions
Liu, H., Wilson, I. B., Goggin, K., Reynolds, N. R., Simoni, J. M., Golin, C. E., Rosen, M. I., Gross, R., Wagner, G. J., Remien, R. H., et al. (2012) · AIDS and Behavior · doi:10.1007/s10461-012-0272-4 · Cited by 12
“Meta-analyses with individual participant data (IPD) gathered from the constituent studies allows researchers to perform subgroup analyses not conducted by the original studies and to add new information to the data sets by using different statistical methods [22].”
MethodLavallee, J. P., Gíusto, B. D., & Yu, T. (2019) · Climatic Change · doi:10.1007/s10584-019-02402-z · Cited by 14
Temple, J., Salmon, P., Smith, C. T., Huntley, C. D., Byrne, A., & Fisher, P. (2020) · Clinical Psychology Review · doi:10.1016/j.cpr.2020.101883 · Cited by 14
A Nonparametric, Multiple Imputation-Based Method for the Retrospective Integration of Data Sets
Carrig, M. M., Manrique‐Vallier, D., Ranby, K. W., Reiter, J. P., & Hoyle, R. H. (2015) · Multivariate Behavioral Research · doi:10.1080/00273171.2015.1022641 · Cited by 17
“…raw data are available (Glass, 2000), in part because the integration of individual-level data sets allows for the conduct of subgroup analyses not originally performed by the investigators and tests of within-and between-study moderators (Brown et al., 2011; Cooper & Patall, 2009).”
BackgroundMethodMistiaen, P., Jolley, D. J., McGowan, S., Hickey, M. B., Spreeuwenberg, P. M. M., & Francke, A. L. (2010) · The Medical Journal of Australia · doi:10.5694/j.1326-5377.2010.tb04091.x · Cited by 11
Büscher, R., Beisemann, M., Doebler, P., Steubl, L., Domhardt, M., Cuijpers, P. J., Kerkhof, A. J. F. M., & Sander, L. (2020) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph17145179 · Cited by 13
Metadata
- Authors
- Cooper, H., & Patall, E. A.
- Affiliations
- Duke University (US)
- Venue
- Psychological Methods
- Published
- 2009
- Volume
- 14
- Issue
- 2
- Pages
- 165-176
- ISSN
- 1082-989X (ISSN-L)
- 1939-1463
- OpenAlex
- W2003323761
- Semantic Scholar
- ea908786103694ec9857ffd3dd31530573436cdc
- Cited by
- 339
- References
- 43
- Citations by year
- 2026: 10 · 2025: 25 · 2024: 18 · 2023: 21 · 2022: 30 · 2021: 22 · 2020: 22 · 2019: 20 · 2018: 24 · 2017: 16 · 2016: 26 · 2015: 32 · 2014: 17 · 2013: 16 · 2012: 14
- FWCI
- 16.13
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Psychological Methods (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Keywords
- individual patient data, aggregated data, meta-analysis, subgroup analysis, ecological fallacy, Simpson's paradox, causal inference, effect size, data availability
- MeSH terms
- Data Collection, Data Interpretation, Statistical, Humans, Meta-Analysis as Topic, Models, Statistical
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.