The DOI matches its Crossref record; 64 of the 66 references it cites were found in a registry or an index.
Multilevel Meta-Analysis: A Comparison with Traditional Meta-Analytical Procedures
Abstract
In this article, the authors compare the multilevel meta-analysis approach with the more traditional meta-analytical approaches. After a description and comparison of the under-lying models and some of the major techniques, the results of the multilevel approach are compared with those of the traditional approaches, using a simulation study. The results of the simulation study suggest that the maximum likelihood multilevel approach is in general superior to the fixed-effects approaches, unless only a small number of studies is available. For models without moderators, the results of the multilevel approach, however, are not substantially different from the results of the traditional random-effects approaches.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“we used a multilevel approach to metaanalysis (Assink & Wibbelink, 2016), which is a strong method for dealing with dependency of effect sizes, as it accounts for the hierarchical structure of the data in which effect sizes are nested within studies (Van Den Noortgate & Onghena, 2003).”
From Associations between rejection sensitivity and mental health outcomes: A meta-analytic review (2017) · cites this paper as Van Den Noortgate & Onghena, 2003
In the citing paper
“This model is often used for multilevel meta‐analyses, and is generally considered superior to the fixed‐effects approaches used in traditional meta‐analyses (Van Den Noortgate & Onghena, 2003).”
From Does Natural Mentoring Matter? A Multilevel Meta‐analysis on the Association Between Natural Mentoring and Youth Outcomes (2018) · cites this paper as Van Den Noortgate & Onghena, 2003
In the citing paper
“Van den Noortgate and Onghena (2003) compared multilevel meta-analysis with traditional meta-analytic methods and concluded that maximum likelihood multilevel approach is in general superior to the fixed-effects approaches and that the results of the multilevel approach are not substantially different from the results of the traditional random-effects approaches for intercept only models.”
From A Meta-analysis of Attachment to Parents and Delinquency (2012) · cites this paper as Van den Noortgate and Onghena (2003)
In the citing paper
“To deal with dependency of study results, we used a multilevel random effects model for the calculation of combined effect sizes and moderator-analyses (Hox 2002; Van den Noortgate and Onghena 2003).”
From A Meta-analysis of Attachment to Parents and Delinquency (2012) · cites this paper as Van den Noortgate and Onghena 2003
In the citing paper
“The multilevel approach accounts for the hierarchical structure of the data, in which effect sizes are nested within the studies (Van Den Noortgate & Onghena, 2003).”
From The effects of physical activity interventions on psychosocial outcomes in adolescents: a meta-analytic review (2016) · cites this paper as Van Den Noortgate & Onghena, 2003
Citations
Cites
Meta-analysis in clinical trials
DerSimonian, R., & Laird, N. (1986) · Controlled Clinical Trials · doi:10.1016/0197-2456(86)90046-2 · Cited by 39,552
Hierarchical Linear Models: Applications and Data Analysis Methods.
Longford, N. T., Bryk, A. S., & Raudenbush, S. W. (1993) · Contemporary Sociology · doi:10.2307/2075823 · Cited by 18,724
Multilevel analysis : an introduction to basic and advanced multilevel modeling
Snijders, T. A. B., & Bosker, R. (1999) · Cited by 13,786
Statistical Methods for Meta-Analysis
Wasserman, S., Hedges, L. V., & Olkin, I. (1988) · Journal of Educational Statistics · doi:10.2307/1164953 · Cited by 10,326
Distribution Theory for Glass's Estimator of Effect Size and Related Estimators
Hedges, L. V. (1981) · Journal of Educational Statistics · doi:10.2307/1164588 · Cited by 5,695
Methods of Meta-Analysis: Correcting Error and Bias in Research Findings
Schmidt, F. L., & Hunter, J. E. (2015) · doi:10.4135/9781483398105 · Cited by 5,542
The combination of estimates from different experiments
Cochran, W. G. (1954) · Biometrics · doi:10.2307/3001666 · Cited by 5,380
(2006) · Technometrics · doi:10.1198/tech.2006.s417 · Cited by 4,950
Primary, Secondary, and Meta-Analysis of Research
GLASS, G. V. (1976) · Educational Researcher · doi:10.3102/0013189x005010003 · Cited by 4,890
Meta-Analytic Procedures for Social Research
Rosenthal, R. (1991) · doi:10.4135/9781412984997 · Cited by 4,635
Research works printed but not linked: 5
- 10.1037/0033-2909.93.2.388 · doi:10.1037/0033-2909.93.2.388 · Paper page
- Meijer, E., van der Leeden, R. & Busing, F. M. T. A. (1995). Implementing the bootstrap for multilevel models. Multilevel Modelling Newsletter, 7(2), 7-11.
- Mok, M. (1995). Sample size requirements for 2-level designs in educational research. Multilevel Modelling Newsletter, 7(2), 11-15.
- 10.1037/1082-989x.3.3.354 · doi:10.1037/1082-989x.3.3.354 · Paper page
- 10.1037/0033-2909.92.2.500 · doi:10.1037/0033-2909.92.2.500 · Paper page
Cited by
de Ridder, D. T. D., Lensvelt-Mulders, G., Finkenauer, C., Stok, F. M., & Baumeister, R. F. (2012) · Personality and Social Psychology Review · doi:10.1177/1088868311418749 · Cited by 1,693
Fitting three-level meta-analytic models in R: A step-by-step tutorial
Assink, M., & Wibbelink, C. J. M. (2016) · The Quantitative Methods for Psychology · doi:10.20982/tqmp.12.3.p154 · Cited by 1,330
Three-level meta-analysis of dependent effect sizes
Van den Noortgate, W., López-López, J. A., Marín-Martínez, F., & Sánchez-Meca, J. (2013) · Behavior Research Methods · doi:10.3758/s13428-012-0261-6 · Cited by 1,130
“…such as the method of moments in DerSimonian and Laird (1986), will give very similar results, even if the normality assumption that is made for the maximum likelihood procedure is violated (López-lópez, Viechtbauer, Sánchez-Meca, & Marín-Martínez, 2010; Van den Noortgate & Onghena, 2003b).”
BackgroundMethodPrati, G., & Pietrantoni, L. (2009) · Journal of Loss and Trauma · doi:10.1080/15325020902724271 · Cited by 978
Weisz, J. R., Kuppens, S., Ng, M. Y., Eckshtain, D., Ugueto, A. M., Vaughn-Coaxum, R., Jensen-Doss, A., Hawley, K. M., Krumholz Marchette, L. S., Chu, B. C., et al. (2017) · American Psychologist · doi:10.1037/a0040360 · Cited by 862
Metadata
- Authors
- Van Den Noortgate, W., & Onghena, P.
- Affiliations
- Katholieke Universiteit Leuven, Belgium
- KU Leuven (BE)
- ORCID
- Onghena, Patrick
- Venue
- Educational and Psychological Measurement
- Publisher
- SAGE Publications
- Published
- 2003-10
- Volume
- 63
- Issue
- 5
- Pages
- 765-790
- ISSN
- 0013-1644 (ISSN-L)
- 1552-3888
- OpenAlex
- W2085396959
- Cited by
- 218
- References
- 66
- Citations by year
- 2026: 11 · 2025: 10 · 2024: 15 · 2023: 15 · 2022: 26 · 2021: 9 · 2020: 16 · 2019: 10 · 2018: 17 · 2017: 14 · 2016: 12 · 2015: 5 · 2014: 12 · 2013: 8 · 2012: 5
- FWCI
- 2.74
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
- Locations
- Educational and Psychological Measurement (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Keywords
- multilevel meta-analysis, fixed effects model, random effects models, moderators, simulation study, maximum likelihood estimation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Multilevel Meta-Analysis: A Comparison with Traditional Meta-Analytical Procedures. https://app.cochat.ai/papers/10.1177/0013164403251027
- The paper
- Van Den Noortgate, W., & Onghena, P. (2003). Multilevel Meta-Analysis: A Comparison with Traditional Meta-Analytical Procedures. Educational and Psychological Measurement. https://doi.org/10.1177/0013164403251027
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.