Modeling dependent effect sizes with three-level meta-analyses: A structural equation modeling approach
The DOI matches its Crossref record; 73 of the 73 references it cites were found in a registry or an index.
Abstract
Meta-analysis is an indispensable tool used to synthesize research findings in the social, educational, medical, management, and behavioral sciences. Most meta-analytic models assume independence among effect sizes. However, effect sizes can be dependent for various reasons. For example, studies might report multiple effect sizes on the same construct, and effect sizes reported by participants from the same cultural group are likely to be more similar than those reported by other cultural groups. This article reviews the problems and common methods to handle dependent effect sizes. The objective of this article is to demonstrate how 3-level meta-analyses can be used to model dependent effect sizes. The advantages of the structural equation modeling approach over the multilevel approach with regard to conducting a 3-level meta-analysis are discussed. This article also seeks to extend the key concepts of Q statistics, I2, and R2 from 2-level meta-analyses to 3-level meta-analyses. The proposed procedures are implemented using the open source metaSEM package for the R statistical environment. Two real data sets are used to illustrate these procedures. New research directions related to 3-level meta-analyses are discussed.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Therefore, this study employed the three-level meta-analysis model proposed by Cheung (2014), which decomposes variance into three levels: Level 1, sampling variance; Level 2, within-study variance across multiple effect sizes from the same study; and Level 3, between-study variance across independent studies.”
From Mindfulness- and acceptance-based interventions for music performance anxiety: a three-level meta-analysis of therapeutic effects, mechanisms, and boundary conditions (2026) · cites this paper as Cheung, 2014
In the citing paper
“A major advantage of this three-level approach to meta-analysis is that all relevant effects reported in each primary study can be included, implying that all relevant information is preserved.”
From Risk Factors for School Absenteeism and Dropout: A Meta-Analytic Review (2019) · cites this paper as Cheung 2014
In the citing paper
“As we included five studies that provided two effect sizes, this violates the normal requirement of independent effect size measures in meta-analysis (Cheung, 2014).”
From Therapeutic games to reduce anxiety and depression in young people: A systematic review and exploratory meta‐analysis of their use and effectiveness. (2024) · cites this paper as Cheung, 2014
In the citing paper
“To account for the dependency of estimates at multiple times within studies, we fit three-level meta-analytic models with time periods (short-term or long-term) as a moderator.26”
From Adherence to PCSK9 Inhibitors in Clinical Practice (2026) · cites this paper as [26]
In the citing paper
“We thus adopted a three-level meta-analytic model in which effect sizes are nested within experiments, so we explicitly took the dependencies in our data into account in the model63,64.”
From When combinations of humans and AI are useful: A systematic review and meta-analysis (2024) · cites this paper as [64]
3 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Quantifying heterogeneity in a meta‐analysis
Higgins, J. P. T., & Thompson, S. G. (2002) · Statistics in Medicine · doi:10.1002/sim.1186 · Cited by 36,763
Hierarchical Linear Models: Applications and Data Analysis Methods.
Longford, N. T., Bryk, A. S., & Raudenbush, S. W. (1993) · Contemporary Sociology · doi:10.2307/2075823 · Cited by 18,724
mice: Multivariate Imputation by Chained Equations in R
Buuren, S. V., & Groothuis-Oudshoorn, K. (2011) · Journal of Statistical Software · doi:10.18637/jss.v045.i03 · Cited by 13,870 · cited 1×
“Second, conducting MI for multilevel data is challenging (cf. van Buuren & Groothuis-Oudshoorn, 2011).”
MethodMultilevel analysis : an introduction to basic and advanced multilevel modeling
Snijders, T. A. B., & Bosker, R. (1999) · Cited by 13,775
Missing data: our view of the state of the art
Schafer, J. L., & Graham, J. W. (2002) · Psychological Methods · doi:10.1037/1082-989x.7.2.147 · Cited by 11,638
Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2021) · doi:10.1002/9781119558378 · Cited by 9,799
Mixed-Effects Models in S and S-PLUS
(2001) · Technometrics · doi:10.1198/tech.2001.s574 · Cited by 9,278
(2010) · Cited by 7,034
A basic introduction to fixed-effect and random-effects models for meta-analysis
Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2010) · Research Synthesis Methods · doi:10.1002/jrsm.12 · Cited by 6,793 · cited 1×
“On the other hand, random-effects models allow for unconditional inferences to be generalized beyond the studies included under the assumption that the studies were randomly selected (see Borenstein et al., 2010; Hedges & Vevea, 1998).”
MethodMethods of Meta-Analysis: Correcting Error and Bias in Research Findings
Schmidt, F. L., & Hunter, J. E. (2015) · doi:10.4135/9781483398105 · Cited by 5,542
Cited by
Fitting three-level meta-analytic models in R: A step-by-step tutorial
Assink, M., & Wibbelink, C. J. M. (2016) · The Quantitative Methods for Psychology · doi:10.20982/tqmp.12.3.p154 · Cited by 1,330
The relation between short-term emotion dynamics and psychological well-being: A meta-analysis.
Houben, M., Van Den Noortgate, W., & Kuppens, P. (2015) · Psychological Bulletin · doi:10.1037/a0038822 · Cited by 1,043
Sexual assault victimization and psychopathology: A review and meta-analysis
Dworkin, E. R., Menon, S. V., Bystrynski, J., & Allen, N. E. (2017) · Clinical Psychology Review · doi:10.1016/j.cpr.2017.06.002 · Cited by 825
Mertens, S., Herberz, M., Hahnel, U. J. J., & Brosch, T. (2022) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2107346118 · Cited by 738
metaSEM: an R package for meta-analysis using structural equation modeling
Cheung, M. W.-L. (2015) · Frontiers in Psychology · doi:10.3389/fpsyg.2014.01521 · Cited by 685
Self-citationBullying and Suicidal Ideation and Behaviors: A Meta-Analysis
Holt, M. K., Vivolo-Kantor, A. M., Polanin, J. R., Holland, K. M., DeGue, S., Matjasko, J. L., Wolfe, M., & Reid, G. (2015) · Pediatrics · doi:10.1542/peds.2014-1864 · Cited by 675
A Guide to Conducting a Meta-Analysis with Non-Independent Effect Sizes
Cheung, M. W.-L. (2019) · Neuropsychology Review · doi:10.1007/s11065-019-09415-6 · Cited by 610
Self-citationRisk Factors for School Absenteeism and Dropout: A Meta-Analytic Review
Gubbels, J., van der Put, C. E., & Assink, M. (2019) · Journal of Youth and Adolescence · doi:10.1007/s10964-019-01072-5 · Cited by 596
“Because most studies reported on more than one risk factor for school absenteeism or dropout, a traditional random effects (two-level) model was extended to a three-level random effects model (Cheung 2014; Houben et al. 2015; Van den Noortgate et al. 2013, 2014).”
Meta-evaluation of meta-analysis: ten appraisal questions for biologists
Nakagawa, S., Noble, D. W. A., Senior, A. M., & Lagisz, M. (2017) · BMC Biology · doi:10.1186/s12915-017-0357-7 · Cited by 579
The effectiveness of celebrity endorsements: a meta-analysis
Knoll, J., & Matthes, J. (2017) · Journal of the Academy of Marketing Science · doi:10.1007/s11747-016-0503-8 · Cited by 601
“The same deficits apply to shifting the unit of analysis (Cheung 2014).”
BackgroundMethodWhen combinations of humans and AI are useful: A systematic review and meta-analysis
Vaccaro, M., Almaatouq, A., & Malone, T. (2024) · Nature Human Behaviour · doi:10.1038/s41562-024-02024-1 · Cited by 570
“We thus adopted a three-level meta-analytic model in which effect sizes are nested within experiments, so we explicitly took the dependencies in our data into account in the model 63,64 .”
Steinmetz, H., Knappstein, M., Ajzen, I., Schmidt, P., & Kabst, R. (2016) · Zeitschrift für Psychologie · doi:10.1027/2151-2604/a000255 · Cited by 558
Effects of music interventions on stress-related outcomes: a systematic review and two meta-analyses
de Witte, M., Spruit, A., van Hooren, S., Moonen, X., & Stams, G.-J. (2020) · Health Psychology Review · doi:10.1080/17437199.2019.1627897 · Cited by 539
Which Healthy Eating Nudges Work Best? A Meta-Analysis of Field Experiments
Cadario, R., & Chandon, P. (2020) · Marketing Science · doi:10.1287/mksc.2018.1128 · Cited by 504
Yeager, D. S., Fong, C. J., Lee, H. Y., & Espelage, D. L. (2015) · Journal of Applied Developmental Psychology · doi:10.1016/j.appdev.2014.11.005 · Cited by 472
Wang, M.-T., L. Degol, J., Amemiya, J., Parr, A., & Guo, J. (2020) · Developmental Review · doi:10.1016/j.dr.2020.100912 · Cited by 470
“This integrated modeling approach provides additional methodological advantages, such as flexible parameter constraints, a greater accuracy for confidence intervals, and the use of full information maximum likelihood for handling missing covariates (Cheung, 2009, 2014).”
BackgroundMethodInfluentialMeta-analysis of multiple outcomes: a multilevel approach
Van den Noortgate, W., López-López, J. A., Marín-Martínez, F., & Sánchez-Meca, J. (2015) · Behavior Research Methods · doi:10.3758/s13428-014-0527-2 · Cited by 450
Music therapy for stress reduction: a systematic review and meta-analysis
de Witte, M., Pinho, A. D. S., Stams, G.-J., Moonen, X., Bos, A. E. R., & van Hooren, S. (2022) · Health Psychology Review · doi:10.1080/17437199.2020.1846580 · Cited by 444
“We applied a multilevel approach to meta-analysis in order to account for the interdependency of effect sizes (see Assink et al., 2015; Cheung, 2014; de Witte et al., 2020a; Houben et al., 2015; Spruit et al., 2016).”
MethodZeegers, M. A. J., Colonnesi, C., Stams, G.-J. J. M., & Meins, E. (2017) · Psychological Bulletin · doi:10.1037/bul0000114 · Cited by 419
Disclosure of sensitive behaviors across self-administered survey modes: a meta-analysis
Gnambs, T., & Kaspar, K. (2015) · Behavior Research Methods · doi:10.3758/s13428-014-0533-4 · Cited by 422
The Stability and Change of Loneliness Across the Life Span: A Meta-Analysis of Longitudinal Studies
Mund, M., Freuding, M. M., Möbius, K., Horn, N., & Neyer, F. J. (2020) · Personality and Social Psychology Review · doi:10.1177/1088868319850738 · Cited by 417
Handling Complex Meta-analytic Data Structures Using Robust Variance Estimates: a Tutorial in R
Tanner-Smith, E. E., Tipton, E., & Polanin, J. R. (2016) · Journal of Developmental and Life-Course Criminology · doi:10.1007/s40865-016-0026-5 · Cited by 390
Donald, J. N., Sahdra, B. K., Van Zanden, B., Duineveld, J. J., Atkins, P. W. B., Marshall, S. L., & Ciarrochi, J. (2019) · British Journal of Psychology · doi:10.1111/bjop.12338 · Cited by 371
The Relation Between Family Socioeconomic Status and Academic Achievement in China: A Meta-analysis
Liu, J., Peng, P., & Luo, L. (2020) · Educational Psychology Review · doi:10.1007/s10648-019-09494-0 · Cited by 381
Immigration and Crime: Assessing a Contentious Issue
Ousey, G. C., & Kubrin, C. E. (2018) · Annual Review of Criminology · doi:10.1146/annurev-criminol-032317-092026 · Cited by 381
Evaluating the Impact of Attempts to Correct Health Misinformation on Social Media: A Meta-Analysis
Walter, N., Brooks, J. J., Saucier, C. J., & Suresh, S. (2021) · Health Communication · doi:10.1080/10410236.2020.1794553 · Cited by 374
de Jong, K., Conijn, J. M., Gallagher, R. A. V., Reshetnikova, A. S., Heij, M., & Lutz, M. C. (2021) · Clinical Psychology Review · doi:10.1016/j.cpr.2021.102002 · Cited by 417
The Effects of Youth Mentoring Programs: A Meta-analysis of Outcome Studies
Raposa, E. B., Rhodes, J., Stams, G. J. J. M., Card, N., Burton, S., Schwartz, S., Sykes, L. A. Y., Kanchewa, S., Kupersmidt, J., & Hussain, S. (2019) · Journal of Youth and Adolescence · doi:10.1007/s10964-019-00982-8 · Cited by 366
Risk factors for child neglect: A meta-analytic review
Mulder, T. M., Kuiper, K. C., van der Put, C. E., Stams, G.-J. J. M., & Assink, M. (2018) · Child Abuse & Neglect · doi:10.1016/j.chiabu.2018.01.006 · Cited by 366
Hennessy, E. A., & Johnson, B. T. (2020) · Research Synthesis Methods · doi:10.1002/jrsm.1390 · Cited by 348
Spiegel, J. A., Goodrich, J. M., Morris, B. M., Osborne, C. M., & Lonigan, C. J. (2021) · Psychological Bulletin · doi:10.1037/bul0000322 · Cited by 362
Cuijpers, P., Miguel, C., Harrer, M., Plessen, C. Y., Ciharova, M., Ebert, D., & Karyotaki, E. (2023) · World Psychiatry · doi:10.1002/wps.21069 · Cited by 354
Risk factors for persistent delinquent behavior among juveniles: A meta-analytic review
Assink, M., van der Put, C. E., Hoeve, M., de Vries, S. L. A., Stams, G. J. J. M., & Oort, F. J. (2015) · Clinical Psychology Review · doi:10.1016/j.cpr.2015.08.002 · Cited by 338
Associations between rejection sensitivity and mental health outcomes: A meta-analytic review
Gao, S., Assink, M., Cipriani, A., & Lin, K. (2017) · Clinical Psychology Review · doi:10.1016/j.cpr.2017.08.007 · Cited by 330
Risk factors for child sexual abuse victimization: A meta-analytic review.
Assink, M., van der Put, C. E., Meeuwsen, M. W. C. M., de Jong, N. M., Oort, F. J., Stams, G. J. J. M., & Hoeve, M. (2019) · Psychological bulletin · doi:10.1037/bul0000188 · Cited by 322
The big five, the huge two, and creative self-beliefs: A meta-analysis.
Karwowski, M., & Lebuda, I. (2016) · Psychology of Aesthetics, Creativity, and the Arts · doi:10.1037/aca0000035 · Cited by 315
van de Wetering, J., Leijten, P., Spitzer, J., & Thomaes, S. (2022) · Journal of Environmental Psychology · doi:10.1016/j.jenvp.2022.101782 · Cited by 289
Complex agricultural landscapes host more biodiversity than simple ones: A global meta-analysis
Estrada-Carmona, N., Sánchez, A. C., Remans, R., & Jones, S. K. (2022) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2203385119 · Cited by 284
Detecting Selection Bias in Meta-Analyses with Multiple Outcomes: A Simulation Study
Fernández-Castilla, B., Declercq, L., Jamshidi, L., Beretvas, S. N., Onghena, P., & Van den Noortgate, W. (2021) · The Journal of Experimental Education · doi:10.1080/00220973.2019.1582470 · Cited by 285
Tomaso, C. C., Johnson, A. B., & Nelson, T. D. (2021) · SLEEP · doi:10.1093/sleep/zsaa289 · Cited by 282
Spruit, A., Goos, L., Weenink, N., Rodenburg, R., Niemeyer, H., Stams, G. J., & Colonnesi, C. (2020) · Clinical Child and Family Psychology Review · doi:10.1007/s10567-019-00299-9 · Cited by 280
“The sampling variance of observed effect sizes (level 1) was estimated by using the formula of Cheung (2014).”
BackgroundMethodMoeyaert, M., Ugille, M., Natasha Beretvas, S., Ferron, J., Bunuan, R., & Van den Noortgate, W. (2017) · International Journal of Social Research Methodology · doi:10.1080/13645579.2016.1252189 · Cited by 274
Dissociation in victims of childhood abuse or neglect: a meta-analytic review
Vonderlin, R., Kleindienst, N., Alpers, G. W., Bohus, M., Lyssenko, L., & Schmahl, C. (2018) · Psychological Medicine · doi:10.1017/s0033291718000740 · Cited by 280
Nonindependence and sensitivity analyses in ecological and evolutionary meta‐analyses
Noble, D. W. A., Lagisz, M., O'dea, R. E., & Nakagawa, S. (2017) · Molecular Ecology · doi:10.1111/mec.14031 · Cited by 267
Matthes, J., Knoll, J., & von Sikorski, C. (2018) · Communication Research · doi:10.1177/0093650217745429 · Cited by 269
Scammacca, N., Roberts, G., & Stuebing, K. K. (2014) · Review of Educational Research · doi:10.3102/0034654313500826 · Cited by 267
Moitra, M., Santomauro, D., Degenhardt, L., Collins, P. Y., Whiteford, H., Vos, T., & Ferrari, A. (2021) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2021.02.053 · Cited by 252
van Eldik, W. M., de Haan, A. D., Parry, L. Q., Davies, P. T., Luijk, M. P. C. M., Arends, L. R., & Prinzie, P. (2020) · Psychological Bulletin · doi:10.1037/bul0000233 · Cited by 258
Identifying Effective Components of Child Maltreatment Interventions: A Meta-analysis
van der Put, C. E., Assink, M., Gubbels, J., & Boekhout van Solinge, N. F. (2018) · Clinical Child and Family Psychology Review · doi:10.1007/s10567-017-0250-5 · Cited by 258
Pornography Consumption and Satisfaction: A Meta-Analysis: Pornography and Satisfaction
Wright, P. J., Tokunaga, R. S., Kraus, A., & Klann, E. (2017) · Human Communication Research · doi:10.1111/hcre.12108 · Cited by 257
Spruit, A., Assink, M., van Vugt, E., van der Put, C., & Stams, G. J. (2016) · Clinical Psychology Review · doi:10.1016/j.cpr.2016.03.006 · Cited by 250
Meta-analytic structural equation modeling with moderating effects on SEM parameters.
Jak, S., & Cheung, M. W. L. (2020) · Psychological Methods · doi:10.1037/met0000245 · Cited by 262
Self-citationFoster care placement instability: A meta-analytic review
Konijn, C., Admiraal, S., Baart, J., van Rooij, F., Stams, G.-J., Colonnesi, C., Lindauer, R., & Assink, M. (2019) · Children and Youth Services Review · doi:10.1016/j.childyouth.2018.12.002 · Cited by 253
“The sampling variance of observed effect sizes (Level 1) was estimated by using the formula of Cheung (2014).”
MethodScherrer, V., & Preckel, F. (2019) · Review of Educational Research · doi:10.3102/0034654318819127 · Cited by 251
Mind full of ideas: A meta-analysis of the mindfulness–creativity link
Lebuda, I., Zabelina, D. L., & Karwowski, M. (2016) · Personality and Individual Differences · doi:10.1016/j.paid.2015.09.040 · Cited by 250
Happy Entrepreneurs? Everywhere? A Meta-Analysis of Entrepreneurship and Wellbeing
Stephan, U., Rauch, A., & Hatak, I. (2023) · Entrepreneurship Theory and Practice · doi:10.1177/10422587211072799 · Cited by 245
The association between teacher leadership and student achievement: A meta-analysis
Shen, J., Wu, H., Reeves, P., Zheng, Y., Ryan, L., & Anderson, D. (2020) · Educational Research Review · doi:10.1016/j.edurev.2020.100357 · Cited by 238
A Guide to Conducting a Meta-Analysis
Cheung, M. W.-L., & Vijayakumar, R. (2016) · Neuropsychology Review · doi:10.1007/s11065-016-9319-z · Cited by 239
Self-citationThe intergenerational transmission of child maltreatment: A three-level meta-analysis
Assink, M., Spruit, A., Schuts, M., Lindauer, R., van der Put, C. E., & Stams, G.-J. J. M. (2018) · Child Abuse & Neglect · doi:10.1016/j.chiabu.2018.07.037 · Cited by 275
Li, J.-B., Dou, K., & Liang, Y. (2021) · Journal of Happiness Studies · doi:10.1007/s10902-020-00230-y · Cited by 233
Promoting memory consolidation during sleep: A meta-analysis of targeted memory reactivation
Hu, X., Cheng, L. Y., Chiu, M. H., & Paller, K. A. (2020) · Psychological Bulletin · doi:10.1037/bul0000223 · Cited by 232
“To examine how much effect sizes varied from each other in the multilevel model, we used Cochran’s Q statistic to test whether individual effect size would vary significantly across the whole dataset (i.e., heterogeneity, Borenstein et al., 2011; Cheung, 2014).”
Kirby, J. N., Day, J., & Sagar, V. (2019) · Clinical Psychology Review · doi:10.1016/j.cpr.2019.03.001 · Cited by 232
Fiction reading has a small positive impact on social cognition: A meta-analysis.
Dodell-Feder, D., & Tamir, D. I. (2018) · Journal of Experimental Psychology: General · doi:10.1037/xge0000395 · Cited by 223
van Dijk, R., van der Valk, I. E., Deković, M., & Branje, S. (2020) · Clinical Psychology Review · doi:10.1016/j.cpr.2020.101861 · Cited by 217
Boele, S., Van der Graaff, J., de Wied, M., Van der Valk, I. E., Crocetti, E., & Branje, S. (2019) · Journal of Youth and Adolescence · doi:10.1007/s10964-019-00993-5 · Cited by 216
“Signi fi cance of moderators was tested with an omnibus test (i.e., the Q m statistic) (Cheung 2014).”
Szumski, G., Smogorzewska, J., & Karwowski, M. (2017) · Educational Research Review · doi:10.1016/j.edurev.2017.02.004 · Cited by 214
Nakagawa, S., Yang, Y., Macartney, E. L., Spake, R., & Lagisz, M. (2023) · Environmental Evidence · doi:10.1186/s13750-023-00301-6 · Cited by 210
Fixed- and random-effects meta-analytic structural equation modeling: Examples and analyses in R
Cheung, M. W.-L. (2014) · Behavior Research Methods · doi:10.3758/s13428-013-0361-y · Cited by 208
Self-citationvan Aar, J., Leijten, P., Orobio de Castro, B., & Overbeek, G. (2017) · Clinical Psychology Review · doi:10.1016/j.cpr.2016.11.006 · Cited by 208
D’Oria, L., Crook, T. R., Ketchen, D. J., Sirmon, D. G., & Wright, M. (2021) · Journal of Management · doi:10.1177/0149206321994182 · Cited by 205
Creativity and Well‐being: A Meta‐analysis
Acar, S., Tadik, H., Myers, D., van der Sman, C., & Uysal, R. (2021) · The Journal of Creative Behavior · doi:10.1002/jocb.485 · Cited by 208
Maes, M., Nelemans, S. A., Danneel, S., Fernández-Castilla, B., Van den Noortgate, W., Goossens, L., & Vanhalst, J. (2019) · Developmental Psychology · doi:10.1037/dev0000719 · Cited by 207
The relation between students’ socioeconomic status and ICT literacy: Findings from a meta-analysis
Scherer, R., & Siddiq, F. (2019) · Computers & Education · doi:10.1016/j.compedu.2019.04.011 · Cited by 202
The cognitive benefits of learning computer programming: A meta-analysis of transfer effects.
Scherer, R., Siddiq, F., & Sánchez Viveros, B. (2019) · Journal of Educational Psychology · doi:10.1037/edu0000314 · Cited by 205
Sexualizing Media Use and Self-Objectification: A Meta-Analysis
Karsay, K., Knoll, J., & Matthes, J. (2018) · Psychology of Women Quarterly · doi:10.1177/0361684317743019 · Cited by 201
Johnson, B. T., & Hennessy, E. A. (2019) · Social Science & Medicine · doi:10.1016/j.socscimed.2019.05.035 · Cited by 201
Alternative metrics in scientometrics: a meta-analysis of research into three altmetrics
Bornmann, L. (2015) · Scientometrics · doi:10.1007/s11192-015-1565-y · Cited by 202
He is a Stud, She is a Slut! A Meta-Analysis on the Continued Existence of Sexual Double Standards
Endendijk, J. J., van Baar, A. L., & Deković, M. (2020) · Personality and Social Psychology Review · doi:10.1177/1088868319891310 · Cited by 200
“…dependency between effect sizes that come from the same study, because it partitions sources of variance: variance between studies, variance between samples from the same study, variance between effect sizes from the same study, and sampling variance (Cheung, 2014; Van den Noortgate et al., 2013).”
The Stressful Personality: A Meta-Analytical Review of the Relation Between Personality and Stress
Luo, J., Zhang, B., Cao, M., & Roberts, B. W. (2023) · Personality and Social Psychology Review · doi:10.1177/10888683221104002 · Cited by 170
Cai, Z., Mao, P., Wang, Z., Wang, D., He, J., & Fan, X. (2023) · Adolescent Research Review · doi:10.1007/s40894-022-00201-9 · Cited by 192
Fernández-Castilla, B., Jamshidi, L., Declercq, L., Beretvas, S. N., Onghena, P., & Van den Noortgate, W. (2020) · Behavior Research Methods · doi:10.3758/s13428-020-01373-9 · Cited by 181
Fairbairn, C. E., Briley, D. A., Kang, D., Fraley, R. C., Hankin, B. L., & Ariss, T. (2018) · Psychological Bulletin · doi:10.1037/bul0000141 · Cited by 182
“…in the current review is our use of meta-analytic models that account for clustered standard errors and thus allow for the estimation of effects on the basis of multiple effects sizes per study (Briley & Tucker-Drob, 2014; Cheung, 2014; Luhmann, Hofmann, Eid, & Lucas, 2012; Roberts et al., 2017).”
A Meta-Analysis of Interpersonal and Organizational Workplace Deviance Research
Mackey, J. D., McAllister, C. P., Ellen, B. P., & Carson, J. E. (2021) · Journal of Management · doi:10.1177/0149206319862612 · Cited by 181
Batastini, A. B., Paprzycki, P., Jones, A. C. T., & MacLean, N. (2021) · Clinical Psychology Review · doi:10.1016/j.cpr.2020.101944 · Cited by 181
Donald, J. N., Bradshaw, E. L., Ryan, R. M., Basarkod, G., Ciarrochi, J., Duineveld, J. J., Guo, J., & Sahdra, B. K. (2020) · Personality and Social Psychology Bulletin · doi:10.1177/0146167219896136 · Cited by 182
The effects of educational robotics in STEM education: a multilevel meta-analysis
Ouyang, F., & Xu, W. (2024) · International Journal of STEM Education · doi:10.1186/s40594-024-00469-4 · Cited by 171
Parenting and Self-Control Across Early to Late Adolescence: A Three-Level Meta-Analysis
Li, J.-B., Willems, Y. E., Stok, F. M., Deković, M., Bartels, M., & Finkenauer, C. (2019) · Perspectives on Psychological Science · doi:10.1177/1745691619863046 · Cited by 179
Cheng, C., Wang, H.-Y., Sigerson, L., & Chau, C.-L. (2019) · Psychological Bulletin · doi:10.1037/bul0000198 · Cited by 173
Memory and comprehension of narrative versus expository texts: A meta-analysis
Mar, R. A., Li, J., Nguyen, A. T. P., & Ta, C. P. (2021) · Psychonomic Bulletin & Review · doi:10.3758/s13423-020-01853-1 · Cited by 175
Depression and Cognitive Control across the Lifespan: a Systematic Review and Meta-Analysis
Dotson, V. M., McClintock, S. M., Verhaeghen, P., Kim, J. U., Draheim, A. A., Syzmkowicz, S. M., Gradone, A. M., Bogoian, H. R., & Wit, L. D. (2020) · Neuropsychology Review · doi:10.1007/s11065-020-09436-6 · Cited by 173
Overcoming Resistance Through Narratives: Findings from a Meta-Analytic Review
Ratcliff, C. L., & Sun, Y. (2020) · Human Communication Research · doi:10.1093/hcr/hqz017 · Cited by 170
Gnambs, T. (2014) · Journal of Research in Personality · doi:10.1016/j.jrp.2014.06.003 · Cited by 168
Scherer, R., Siddiq, F., & Sánchez Viveros, B. (2020) · Computers in Human Behavior · doi:10.1016/j.chb.2020.106349 · Cited by 164
Cheng, C., Cheung, M. W.-L., & Wang, H.-Y. (2018) · Computers in Human Behavior · doi:10.1016/j.chb.2018.06.033 · Cited by 162
Self-citationorchaRd 2.0: An R package for visualising meta‐analyses with orchard plots
Nakagawa, S., Lagisz, M., O’Dea, R. E., Pottier, P., Rutkowska, J., Senior, A. M., Yang, Y., & Noble, D. W. A. (2023) · Methods in Ecology and Evolution · doi:10.1111/2041-210x.14152 · Cited by 223
Physical Activity and School Engagement in Youth: A Systematic Review and Meta-Analysis
Owen, K., Parker, P. D., Zanden, B. E. V., MacMillan, F., Astell‐Burt, T., & Lonsdale, C. (2016) · Educational Psychologist · doi:10.1080/00461520.2016.1151793 · Cited by 199
UPPS-P model impulsivity and marijuana use behaviors in adolescents: A meta-analysis
VanderVeen, J. D., Hershberger, A. R., & Cyders, M. A. (2016) · Drug and Alcohol Dependence · doi:10.1016/j.drugalcdep.2016.09.016 · Cited by 179
How Terrorism Does (and Does Not) Affect Citizens’ Political Attitudes: A Meta‐Analysis
Godefroidt, A. (2022) · American Journal of Political Science · doi:10.1111/ajps.12692 · Cited by 161
Dekkers, T. J., van Rentergem, J. A. A., Meijer, B., Popma, A., Wagemaker, E., & Huizenga, H. M. (2018) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2018.12.004 · Cited by 160
Automated feedback and writing: a multi-level meta-analysis of effects on students' performance
Fleckenstein, J., Liebenow, L. W., & Meyer, J. (2023) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2023.1162454 · Cited by 145
“…sizes of the included studies by applying a three-level model with random effects to take into account that several studies of our meta-analysis reported more than one effect size (Geeraert et al., 2004; Konstantopoulos, 2011; Cheung, 2014; Van den Noortgate et al., 2015; Assink and Wibbelink, 2016).”
Method
Metadata
- Authors
- Cheung, M. W.-L.
- Affiliations
- National University of Singapore (SG)
- Corresponding author
- Cheung, Mike W.-L.
- ORCID
- Cheung, Mike W.-L.
- Venue
- Psychological Methods
- Published
- 2014
- Volume
- 19
- Issue
- 2
- Pages
- 211-229
- ISSN
- 1082-989X (ISSN-L)
- 1939-1463
- OpenAlex
- W2122962297
- Semantic Scholar
- 80381b0d516d07e42e308f983624bffd497b1d9b
- Cited by
- 1,342
- References
- 73
- Citations by year
- 2026: 294 · 2025: 203 · 2024: 174 · 2023: 157 · 2022: 143 · 2021: 91 · 2020: 80 · 2019: 69 · 2018: 41 · 2017: 29 · 2016: 27 · 2015: 22 · 2014: 8
- FWCI
- 22.07
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Psychological Methods (publishedVersion)
- PubMed (publishedVersion)
- Funders
- Ministry of Education (R581-000-111-112)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Keywords
- dependent effect sizes, structural equation modeling, multilevel meta-analysis, q-statistics, I2 statistic, cultural similarity, heterogeneity measures
- MeSH terms
- Humans, Models, Psychological, Research Design, Meta-Analysis as Topic, Models, Statistical, Multivariate Analysis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.