Multilevel modeling myths.
The DOI matches its Crossref record; 39 of the 39 references it cites were found in a registry or an index.
Abstract
The use of multilevel modeling (MLM) to analyze nested data has grown in popularity over the years in the study of school psychology. However, with the increase in use, several statistical misconceptions about the technique have also proliferated. We discuss some commonly cited myths and golden rules related to the use of MLM, explain their origin, and suggest approaches to dealing with certain issues. Misunderstandings related to the use of the intraclass correlation, design effects, minimum sample size, multilevel factor structures, model R², and the misestimation of standard errors are reviewed. Many of the cited myths have much truth in them-though at times, researchers may not be aware of the exceptions to the rules that prevent their overall generalization. Although nesting should be accounted for, researchers should realize that MLM, which is a powerful and flexible technique, is not the only method that can be used to account for the clustering effect. (PsycINFO Database Record
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Cohen, J. (1992) · Psychological Bulletin · doi:10.1037//0033-2909.112.1.155 · Cited by 8,771
Sufficient Sample Sizes for Multilevel Modeling
Maas, C. J. M., & Hox, J. J. (2005) · Methodology · doi:10.1027/1614-2241.1.3.86 · Cited by 3,582
(2000) · Cited by 3,573
Worms: Identifying Impacts on Education and Health in the Presence of Treatment Externalities
Miguel, E., & Kremer, M. (2004) · Econometrica · doi:10.1111/j.1468-0262.2004.00481.x · Cited by 2,268
Statistical Methods for the Social Sciences
EJD, Agresti, A., & Finlay, B. (1998) · Journal of the American Statistical Association · doi:10.2307/2670147 · Cited by 2,120
Complex Sample Data in Structural Equation Modeling
Muthen, B. O., & Satorra, A. (1995) · Sociological Methodology · doi:10.2307/271070 · Cited by 1,586
A practical guide to multilevel modeling
Peugh, J. L. (2010) · Journal of School Psychology · doi:10.1016/j.jsp.2009.09.002 · Cited by 1,418
Various Versatile Variances: An Object-Oriented Implementation of Clustered Covariances in R
Zeileis, A., Köll, S., & Graham, N. (2020) · Journal of Statistical Software · doi:10.18637/jss.v095.i01 · Cited by 687
Applying multilevel confirmatory factor analysis techniques to the study of leadership
Dyer, N. G., Hanges, P. J., & Hall, R. J. (2005) · The Leadership Quarterly · doi:10.1016/j.leaqua.2004.09.009 · Cited by 537
A Primer on Multilevel Modeling
Hayes, A. F. (2006) · Human Communication Research · doi:10.1111/j.1468-2958.2006.00281.x · Cited by 530
Parental Involvement and Students' Academic Achievement: A Growth Modeling Analysis
Fan, X. (2001) · The Journal of Experimental Education · doi:10.1080/00220970109599497 · Cited by 489
Dalal, D. K., & Zickar, M. J. (2012) · Organizational Research Methods · doi:10.1177/1094428111430540 · Cited by 469
An Introduction to Multilevel Modeling for Social and Personality Psychology
Nezlek, J. B. (2008) · Social and Personality Psychology Compass · doi:10.1111/j.1751-9004.2007.00059.x · Cited by 433
Bliese, P. D., & Hanges, P. J. (2004) · Organizational Research Methods · doi:10.1177/1094428104268542 · Cited by 412
Maas, C. J. M., & Hox, J. J. (2004) · Computational Statistics & Data Analysis · doi:10.1016/j.csda.2003.08.006 · Cited by 399
Work stressors, role-based performance, and the moderating influence of organizational support.
Wallace, J. C., Edwards, B. D., Arnold, T., Frazier, M. L., & Finch, D. M. (2009) · Journal of Applied Psychology · doi:10.1037/a0013090 · Cited by 309
Alternatives to Multilevel Modeling for the Analysis of Clustered Data
Huang, F. L. (2016) · The Journal of Experimental Education · doi:10.1080/00220973.2014.952397 · Cited by 283
Self-citationExplained variance measures for multilevel models
LaHuis, D. M., Hartman, M. J., Hakoyama, S., & Clark, P. C. (2014) · Organizational Research Methods · doi:10.1177/1094428114541701 · Cited by 239
Vandenberghe, C., Bentein, K., Michon, R., Chebat, J.-C., Tremblay, M., & Fils, J.-F. (2007) · Journal of Applied Psychology · doi:10.1037/0021-9010.92.4.1177 · Cited by 200
Modeling Certainty with Clustered Data: A Comparison of Methods
Arceneaux, K., & Nickerson, D. W. (2009) · Political Analysis · doi:10.1093/pan/mpp004 · Cited by 196
School climate, family structure, and academic achievement: A study of moderation effects.
O'Malley, M., Voight, A., Renshaw, T. L., & Eklund, K. (2015) · School Psychology Quarterly · doi:10.1037/spq0000076 · Cited by 195
Bell, B. A., Morgan, G. B., Schoeneberger, J. A., Kromrey, J. D., & Ferron, J. M. (2014) · Methodology · doi:10.1027/1614-2241/a000062 · Cited by 193
Students' perceptions of school climate in the U.S. and China.
Yang, C., Bear, G. G., Chen, F., Zhang, W., Blank, J. C., & Huang, X. (2013) · School Psychology Quarterly · doi:10.1037/spq0000002 · Cited by 179
Dealing With Dependence (Part I): Understanding the Effects of Clustered Data
McCoach, D. B., & Adelson, J. L. (2010) · Gifted Child Quarterly · doi:10.1177/0016986210363076 · Cited by 166
Bear, G. G., Yang, C., Pell, M. M., & Gaskins, C. S. (2014) · Learning Environments Research · doi:10.1007/s10984-014-9162-1 · Cited by 164
Multilevel multi-informant structure of the Authoritative School Climate Survey.
Konold, T., Cornell, D., Huang, F., Meyer, P., Lacey, A., Nekvasil, E., Heilbrun, A., & Shukla, K. (2014) · School Psychology Quarterly · doi:10.1037/spq0000062 · Cited by 154
Self-citationKaplan, D., & Elliott, P. R. (1997) · Structural Equation Modeling: A Multidisciplinary Journal · doi:10.1080/10705519709540056 · Cited by 108
A Bootstrap Method for Conducting Statistical Inference with Clustered Data
Harden, J. J. (2011) · State Politics & Policy Quarterly · doi:10.1177/1532440011406233 · Cited by 108
Cornell, D. G., Allen, K., & Fan, X. (2012) · School Psychology Review · doi:10.1080/02796015.2012.12087378 · Cited by 106
Alderman, H., Konde-Lule, J. K., Sebuliba, I., Bundy, D. A. P., & Hall, A. (2006) · BMJ · doi:10.1136/bmj.38877.393530.7c · Cited by 102
The General Linear Model as Structural Equation Modeling
Graham, J. M. (2007) · Journal of Educational and Behavioral Statistics · doi:10.3102/1076998607306151 · Cited by 89
Niehaus, E. K., Campbell, C. M., & Inkelas, K. K. (2013) · Research in Higher Education · doi:10.1007/s11162-013-9306-7 · Cited by 88
Huang, F. L., & Cornell, D. G. (2015) · Journal of Psychoeducational Assessment · doi:10.1177/0734282915570278 · Cited by 66
Self-citationPreliminary development of a revised version of the School Climate Measure.
Zullig, K. J., Collins, R., Ghani, N., Hunter, A. A., Patton, J., Huebner, E. S., & Zhang, J. (2015) · Psychological Assessment · doi:10.1037/pas0000070 · Cited by 62
van Starrenburg, M. L. A., Kuijpers, R. C. M. W., Kleinjan, M., Hutschemaekers, G. J. M., & Engels, R. C. M. E. (2017) · Prevention Science · doi:10.1007/s11121-016-0725-5 · Cited by 38
Jobs and organisations: Explaining group level differences in job satisfaction in the banking sector
Cambré, B., Kippers, E., van Veldhoven, M. J. P. M., & Witte, H. D. (2012) · Personnel Review · doi:10.1108/00483481211200033 · Cited by 28
Resnicow, K., Zhang, N., VAUGHAN, R. D., Reddy, S. P., James, S., & Murray, D. M. (2010) · American Journal of Public Health · doi:10.2105/ajph.2009.160879 · Cited by 23
Analyzing Group Level Effects with Clustered Data Using Taylor Series Linearization
Huang, F. L. (2014) · Practical assessment, research & evaluation · doi:10.7275/ntnk-d929 · Cited by 19
Self-citationRocconi, L. (2013) · Higher Education · doi:10.1007/s10734-013-9615-y · Cited by 18
Cited by
Soncini, A., Visintin, E. P., Matteucci, M. C., Tomasetto, C., & Butera, F. (2022) · Learning and Instruction · doi:10.1016/j.learninstruc.2022.101627 · Cited by 43
Lindquist, E. G., & Raposa, E. B. (2020) · American Journal of Community Psychology · doi:10.1002/ajcp.12412 · Cited by 4
Metadata
- Authors
- Huang, F. L.
- Venue
- School Psychology Quarterly
- Published
- 2018-09
- Volume
- 33
- Issue
- 3
- Pages
- 492-499
- ISSN
- 1045-3830 (ISSN-L)
- 1939-1560
- OpenAlex
- W2886683942
- Cited by
- 203
- References
- 39
- FWCI
- 11.09
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.