The DOI matches its Crossref record; 73 of the 73 references it cites were found in a registry or an index.
The multilevel latent covariate model: A new, more reliable approach to group-level effects in contextual studies.
Abstract
In multilevel modeling (MLM), group-level (L2) characteristics are often measured by aggregating individual-level (L1) characteristics within each group so as to assess contextual effects (e.g., group-average effects of socioeconomic status, achievement, climate). Most previous applications have used a multilevel manifest covariate (MMC) approach, in which the observed (manifest) group mean is assumed to be perfectly reliable. This article demonstrates mathematically and with simulation results that this MMC approach can result in substantially biased estimates of contextual effects and can substantially underestimate the associated standard errors, depending on the number of L1 individuals per group, the number of groups, the intraclass correlation, the sampling ratio (the percentage of cases within each group sampled), and the nature of the data. To address this pervasive problem, the authors introduce a new multilevel latent covariate (MLC) approach that corrects for unreliability at L2 and results in unbiased estimates of L2 constructs under appropriate conditions. However, under some circumstances when the sampling ratio approaches 100%, the MMC approach provides more accurate estimates. Based on 3 simulations and 2 real-data applications, the authors evaluate the MMC and MLC approaches and suggest when researchers should most appropriately use one, the other, or a combination of both approaches.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Statistical power analysis for the behavioral sciences
(1990) · Computers, Environment and Urban Systems · doi:10.1016/0198-9715(90)90050-4 · Cited by 65,269
(1998) · Cited by 36,049
(2001) · RePEc: Research Papers in Economics · Cited by 27,577
Multilevel analysis : an introduction to basic and advanced multilevel modeling
Snijders, T. A. B., & Bosker, R. (1999) · Cited by 10,963
Statistical Theories of Mental Test Scores
Rozeboom, W. W., Lord, F. M., Novick, M. R., & Birnbaum, A. (1969) · American Educational Research Journal · doi:10.2307/1162101 · Cited by 8,022
Multilevel analysis techniques and applications
Hox, J. J., Moerbeek, M., & van de Schoot, R. (2002) · Cited by 6,735
(2005) · Cited by 6,097
(2006) · Technometrics · doi:10.1198/tech.2006.s417 · Cited by 4,950
Klein, K. J., & Kozlowski, S. W. J. (2000) · Cited by 4,914
Conventional wisdom on measurement: A structural equation perspective.
Bollen, K., & Lennox, R. (1991) · Psychological Bulletin · doi:10.1037/0033-2909.110.2.305 · Cited by 3,618
Sufficient Sample Sizes for Multilevel Modeling
Maas, C. J. M., & Hox, J. J. (2005) · Methodology · doi:10.1027/1614-2241.1.3.86 · Cited by 3,582
(2000) · Cited by 3,573
Gill, J. (2005) · Encyclopedia of Social Measurement · doi:10.1016/b0-12-369398-5/00166-3 · Cited by 3,114
Generalized, Linear, and Mixed Models
McCulloch, C. E., & Searle, S. R. (2000) · Wiley Series in Probability and Statistics · doi:10.1002/0471722073 · Cited by 2,994
Chan, D. K. (1998) · Journal of Applied Psychology · doi:10.1037//0021-9010.83.2.234 · Cited by 2,972
(2016) · Time Series with Mixed Spectra · doi:10.1201/b15154-8 · Cited by 2,739
(2000) · Cited by 2,531
Aldenderfer, M. S., & Blashfield, R. K. (1984) · Cited by 2,358
Aggregation bias in estimates of perceptual agreement.
James, L. R. (1982) · Journal of Applied Psychology · doi:10.1037/0021-9010.67.2.219 · Cited by 2,139
Measurement Error in Nonlinear Models
Carroll, R. J., Ruppert, D., & Stefanski, L. A. (1995) · Springer eBooks · doi:10.1007/978-1-4899-4477-1 · Cited by 1,749
On the nature and direction of relationships between constructs and measures.
Edwards, J. R., & Bagozzi, R. P. (2000) · Psychological Methods · doi:10.1037//1082-989x.5.2.155 · Cited by 1,745
Bauer, D. J., Preacher, K. J., & Gil, K. M. (2006) · Psychological Methods · doi:10.1037/1082-989x.11.2.142 · Cited by 1,704
Generalized Latent Variable Modeling: Multilevel, Longitudinal, and Structural Equation Models
Skrondal, A., & Rabe‐Hesketh, S. (2004) · Thammasat University Digital Collections · Cited by 1,505
Marsh, H. W. (1987) · International Journal of Educational Research · doi:10.1016/0883-0355(87)90001-2 · Cited by 1,333
Self-citationDiscipline and group management in classrooms
Kounin, J. S. (1970) · Medical Entomology and Zoology · Cited by 1,329
Beyond SEM: General latent variable modeling
Muthén, B. O. (2002) · Behaviormetrika · doi:10.2333/bhmk.29.81 · Cited by 1,074
Self-citationLatent Variable Modeling in Heterogeneous Populations
Muthén, B. O. (1989) · Psychometrika · doi:10.1007/bf02296397 · Cited by 1,008
Self-citationMultilevel Models in Educational and Social Research
Tate, R. L., & Goldstein, H. (1988) · Journal of Educational Statistics · doi:10.2307/1164754 · Cited by 993
Journal of Sport and Exercise Psychology
Gill, D. L. (1987) · Journal of Sport Psychology · doi:10.1123/jsp.9.1.1 · Cited by 861
The Effect of Different Forms of Centering in Hierarchical Linear Models
Kreft, I. G. G., de Leeuw, J., & Aiken, L. S. (1995) · Multivariate Behavioral Research · doi:10.1207/s15327906mbr3001_1 · Cited by 850
Generalized Multilevel Structural Equation Modeling
Rabe-Hesketh, S., Skrondal, A., & Pickles, A. (2004) · Psychometrika · doi:10.1007/bf02295939 · Cited by 831
School effects on psychological outcomes during adolescence.
Anderman, E. M. (2002) · Journal of Educational Psychology · doi:10.1037/0022-0663.94.4.795 · Cited by 824
Marsh, H. W., & Hau, K.-T. (2003) · American Psychologist · doi:10.1037/0003-066x.58.5.364 · Cited by 717
Self-citationBeaton, A. E. (1996) · Cited by 612
Separating individual and group effects.
Kenny, D. A., & la Voie, L. (1985) · Journal of Personality and Social Psychology · doi:10.1037//0022-3514.48.2.339 · Cited by 576
Ryan, A. M., Gheen, M. H., & Midgley, C. (1998) · Journal of Educational Psychology · doi:10.1037//0022-0663.90.3.528 · Cited by 527
Davis, J. A. (1966) · American Journal of Sociology · doi:10.1086/224257 · Cited by 524
School Effectiveness and School Improvement: Sustaining Links
Creemers, B. P. M., & Reezigt, G. J. (1997) · School Effectiveness and School Improvement · doi:10.1080/0924345970080402 · Cited by 521
The statistical analysis of data from small groups.
Kenny, D. A., Mannetti, L., Pierro, A., Livi, S., & Kashy, D. A. (2002) · Journal of Personality and Social Psychology · doi:10.1037//0022-3514.83.1.126 · Cited by 469
People are variables too: Multilevel structural equations modeling.
Mehta, P. D., & Neale, M. C. (2005) · Psychological Methods · doi:10.1037/1082-989x.10.3.259 · Cited by 467
TIMSS — Mathematisch-naturwissenschaftlicher Unterricht im internationalen Vergleich
Baumert, J., & Lehmann, R. (1997) · VS Verlag für Sozialwissenschaften eBooks · doi:10.1007/978-3-322-95096-3 · Cited by 440
Reconsidering formative measurement.
Howell, R. D., Breivik, E., & Wilcox, J. B. (2007) · Psychological Methods · doi:10.1037/1082-989x.12.2.205 · Cited by 432
Multilevel Factor Analysis of Class and Student Achievement Components: (671042011-001)
Muthen, B. O. (1990) · PsycEXTRA Dataset · doi:10.1037/e671042011-001 · Cited by 410
A Rule for Inferring Individual-Level Relationships from Aggregate Data
Firebaugh, G. (1978) · American Sociological Review · doi:10.2307/2094779 · Cited by 400
The Scholarship of Teaching and Learning in Higher Education: An Evidence-Based Perspective
(2007) · Springer eBooks · doi:10.1007/1-4020-5742-3 · Cited by 389
Chen, G., Bliese, P. D., & Mathieu, J. E. (2005) · Organizational Research Methods · doi:10.1177/1094428105280056 · Cited by 388
Ecological effects in multi-level studies
Blakely, T. A., & Woodward, A. J. (2000) · Journal of Epidemiology and Community Health · doi:10.1136/jech.54.5.367 · Cited by 385
The Role of Classroom Goal Structure in Students' Use of Self-Handicapping Strategies
Urdan, T., Midgley, C., & Anderman, E. M. (1998) · American Educational Research Journal · doi:10.2307/1163453 · Cited by 366
Marsh, H. W., Kong, C.-K., & Hau, K. (2000) · Journal of Personality and Social Psychology · doi:10.1037//0022-3514.78.2.337 · Cited by 364
Self-citationEffective classroom management and the development of subject-related interest
Kunter, M., Baumert, J., & Köller, O. (2007) · Learning and Instruction · doi:10.1016/j.learninstruc.2007.09.002 · Cited by 360
GONZÁLEZ Y ORTÍZ, F. X. (2003) · Investigación Bibliotecológica: archivonomía, bibliotecología e información · doi:10.22201/iibi.0187358xp.2003.34.4020 · Cited by 330
Croon, M. A., & van Veldhoven, M. J. P. M. (2007) · Psychological Methods · doi:10.1037/1082-989x.12.1.45 · Cited by 276
Research on Classrooms and Schools: Formulation of Questions, Design and Analysis.
Cronbach, L. J. (1976) · Cited by 245
Contextual Analysis: Concepts and Statistical Techniques.
Barton, A. H., Boyd, L. H., & Iversen, G. R. (1981) · Contemporary Sociology · doi:10.2307/2067732 · Cited by 242
Composing Group-Level Constructs From Individual-Level Survey Data
van Mierlo, H., Vermunt, J. K., & Rutte, C. G. (2009) · Organizational Research Methods · doi:10.1177/1094428107309322 · Cited by 238
The Effects of Student Composition on School Outcomes
Harker, R., & Tymms, P. (2004) · School Effectiveness and School Improvement · doi:10.1076/sesi.15.2.177.30432 · Cited by 223
Lüdtke, O., Trautwein, U., Kunter, M., & Baumert, J. (2007) · Learning Environments Research · doi:10.1007/s10984-006-9014-8 · Cited by 211
Self-citationPapaioannou, A., Marsh, H. W., & Theodorakis, Y. (2004) · Journal of Sport and Exercise Psychology · doi:10.1123/jsep.26.1.90 · Cited by 191
Self-citationRaudenbush, S. W., Rowan, B., & Kang, S. J. (1991) · Journal of Educational Statistics · doi:10.2307/1165105 · Cited by 188
Multilevel Methods: Future Directions in Measurement, Longitudinal Analyses, and Nonnormal Outcomes
Bliese, P. D., Chan, D., & Ployhart, R. E. (2007) · Organizational Research Methods · doi:10.1177/1094428107301102 · Cited by 178
Teacher frame of reference and the big-fish–little-pond effect
Lüdtke, O., Köller, O., Marsh, H. W., & Trautwein, U. (2005) · Contemporary Educational Psychology · doi:10.1016/j.cedpsych.2004.10.002 · Cited by 166
Self-citationMarsh, H. W. (2007) · Journal of Educational Psychology · doi:10.1037/0022-0663.99.4.775 · Cited by 157
Self-citationJayasinghe, U. W., Marsh, H. W., & Bond, N. (2003) · Journal of the Royal Statistical Society Series A: Statistics in Society · doi:10.1111/1467-985x.00278 · Cited by 139
Self-citationEstimating the Reliability of Aggregate-Level Variables Based on Individual-Level Characteristics
O’Brien, R. M. (1990) · Sociological Methods & Research · doi:10.1177/0049124190018004004 · Cited by 124
A Novel Bootstrap Procedure for Assessing the Relationship between Class Size and Achievement
Carpenter, J. R., Goldstein, H., & Rasbash, J. (2003) · Journal of the Royal Statistical Society Series C (Applied Statistics) · doi:10.1111/1467-9876.00415 · Cited by 124
Miller, A. D., & Murdock, T. B. (2006) · Contemporary Educational Psychology · doi:10.1016/j.cedpsych.2006.10.006 · Cited by 123
Interjudgmental reliability of reviews for the Journal of Educational Psychology.
Marsh, H. W., & Ball, S. (1981) · Journal of Educational Psychology · doi:10.1037/0022-0663.73.6.872 · Cited by 89
Self-citationUnterricht und schulisches Lernen: Schüler als Quellen der Unterrichtsbeschreibung
Gruehn, S. (2000) · Cited by 64
ANALYSIS OF TWO-LEVEL STRUCTURAL EQUATION MODELS VIA EM TYPE ALGORITHMS
Lee, S., & Poon, W. (1998) · Cited by 49
Constructing Covariates in Multilevel Regression
Asparouhov, T., & Muthén, B. (2007) · Cited by 41
Self-citationLee, S. (2007) · Wiley series in probability and statistics · doi:10.1002/9780470024737.ch3 · Cited by 38
Statistische Artefakte bei Kontexteffekten in der pädagogisch-psychologischen Forschung
Lüdtke, O., Robitzsch, A., & Köller, O. (2002) · Zeitschrift für Pädagogische Psychologie · doi:10.1024//1010-0652.16.34.217 · Cited by 27
Self-citationLongitudinal Analysis With Regressions Among Random Effects: A Latent Variable Modeling Approach
Raykov, T. (2007) · Structural Equation Modeling A Multidisciplinary Journal · doi:10.1080/10705510709336741 · Cited by 24
Cited by
A general multilevel SEM framework for assessing multilevel mediation
Preacher, K. J., Zyphur, M. J., & Zhang, Z. (2010) · Psychological Methods · doi:10.1037/a0020141 · Cited by 3,796
The disaggregation of within-person and between-person effects in longitudinal models of change
Curran, P. J., & Bauer, D. J. (2011) · Annual Review of Psychology · doi:10.1146/annurev.psych.093008.100356 · Cited by 2,056
Hayes, A. F., & Rockwood, N. J. (2020) · American Behavioral Scientist · doi:10.1177/0002764219859633 · Cited by 1,000
Dynamic Structural Equation Models
Asparouhov, T., Hamaker, E. L., & Muthén, B. (2018) · Structural Equation Modeling: A Multidisciplinary Journal · doi:10.1080/10705511.2017.1406803 · Cited by 985
Self-citationAlternative methods for assessing mediation in multilevel data: The advantages of multilevel SEM
Preacher, K. J., Zhang, Z., & Zyphur, M. J. (2011) · Structural Equation Modeling: A Multidisciplinary Journal · doi:10.1080/10705511.2011.557329 · Cited by 889
Metadata
- Authors
- Lüdtke, O., Marsh, H. W., Robitzsch, A., Trautwein, U., Asparouhov, T., & Muthén, B.
- Venue
- Psychological Methods
- Published
- 2008-09
- Volume
- 13
- Issue
- 3
- Pages
- 203-229
- ISSN
- 1082-989X (ISSN-L)
- 1939-1463
- OpenAlex
- W2163995972
- Cited by
- 823
- References
- 73
- FWCI
- 13.35
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.