The Consequence of Ignoring a Level of Nesting in Multilevel Analysis
The DOI matches its Crossref record; 28 of the 28 references it cites were found in a registry or an index.
Abstract
Multilevel analysis is an appropriate tool for the analysis of hierarchically structured data. There may, however, be reasons to ignore one of the levels of nesting in the data analysis. In this article a three level model with one predictor variable is used as a reference model and the top or intermediate level is ignored in the data analysis. Analytical results show that this has an effect on the estimated variance components and that standard errors of regression coefficients estimators may be overestimated, leading to a lower power of the test of the effect of the predictor variable. The magnitude of these results depends on the ignored level and the level at which the predictor variable varies, and on the values of the variance components and the sample sizes.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hierarchical Linear Models: Applications and Data Analysis Methods.
Longford, N. T., Bryk, A. S., & Raudenbush, S. W. (1993) · Contemporary Sociology · doi:10.2307/2075823 · Cited by 18,724 · cited 2×
“Such errors may be avoided by using the multilevel model (Goldstein, 1995; Hox, 2002; Kreft & De Leeuw, 1998; Snijders & Bosker, 1999), sometimes also referred to as hierarchical linear model (Raudenbush & Bryk, 2002) or random coefficient model (Longford, 1993).”
MethodMultilevel analysis : an introduction to basic and advanced multilevel modeling
Snijders, T. A. B., & Bosker, R. (1999) · Cited by 13,923
Cohen, J. (1992) · Psychological Bulletin · doi:10.1037//0033-2909.112.1.155 · Cited by 8,771
Multilevel analysis techniques and applications
Hox, J. J., Moerbeek, M., & van de Schoot, R. (2002) · Cited by 6,810
Self-citationIntroducing Multilevel Modeling
Kreft, I., & de Leeuw, J. (1998) · doi:10.4135/9781849209366 · Cited by 2,997
Hox, J., Moerbeek, M., & van de Schoot, R. (2002) · doi:10.4324/9781410604118 · Cited by 2,419
Self-citationA User's Guide to MLwiN version 2.0
Rasbash, J., Steele, F., Browne, W. J., & Goldstein, H. (2004) · Bristol Research (University of Bristol) · Cited by 1,503
Gulliford, M. C., Ukoumunne, O. C., & Chinn, S. (1999) · American Journal of Epidemiology · doi:10.1093/oxfordjournals.aje.a009904 · Cited by 852
Multilevel mixed linear model analysis using iterative generalized least squares
GOLDSTEIN, H. (1986) · Biometrika · doi:10.1093/biomet/73.1.43 · Cited by 802 · cited 6×
“The estimation method IGLS will be used to explore the effect of ignoring a level of nesting on the estimators of the variance components.”
MethodInfluentialStatistical Modelling Issues in School Effectiveness Studies
Aitkin, M., & Longford, N. (1986) · Journal of the Royal Statistical Society. Series A (General) · doi:10.2307/2981882 · Cited by 790 · cited 2×
“... which means that individuals are nested within clusters, which may themselves be nested within higher order clusters, and so on. Examples of such studies are schoolbased smoking prevention or HIV/STD and pregnancy prevention interventions with pupils nested within schools (Ausems, Mesters, Van Breukelen, & De Vrie…”
BackgroundLongford, N. T. (2011) · International Encyclopedia of Statistical Science · doi:10.1007/978-3-642-04898-2_471 · Cited by 745
Modeled Variance in Two-Level Models
SNIJDERS, T. A. B., & BOSKER, R. J. (1994) · Sociological Methods & Research · doi:10.1177/0049124194022003004 · Cited by 724 · cited 2×
“…since then the 148 MULTIVARIATE BEHAVIORAL RESEARCH variance components, and hence standard errors of regression coefficients estimators, are not only affected by ignoring the level of nesting but also by the removal of these predictor variables from the multilevel model (Snijders & Bosker, 1994).”
MethodStatistical analysis and optimal design for cluster randomized trials.
Raudenbush, S. W. (1997) · Psychological Methods · doi:10.1037/1082-989x.2.2.173 · Cited by 636 · cited 1×
“This rationale should be based on theory, logic or study design (e.g. cluster randomized trials, Raudenbush, 1997).”
MethodBetween- and Within-Cluster Covariate Effects in the Analysis of Clustered Data
Neuhaus, J. M., & Kalbfleisch, J. D. (1998) · Biometrics · doi:10.2307/3109770 · Cited by 525 · cited 2×
“A predictor variable x ijk that varies at more that one level may be split up into orthogonal components that each vary at just one level (Neuhaus & Kalbfleisch, 1998).”
BackgroundMultilevel Modelling of Health Statistics
Leyland, A. H., & Goldstein, H. (2001) · Cited by 449
Opdenakker, M., & Damme, J. V. (2000) · School Effectiveness and School Improvement · doi:10.1076/0924-3453(200006)11:2;1-q;ft165 · Cited by 391
MIXREG: a computer program for mixed-effects regression analysis with autocorrelated errors
Hedeker, D., & Gibbons, R. D. (1996) · Computer Methods and Programs in Biomedicine · doi:10.1016/0169-2607(96)01723-3 · Cited by 258 · cited 3×
“z = ˆˆ /v ar( ) . The computer programs MLwiN (Rasbash et al., 2001) and MIXREG ( Hedeker & Gibbons, 1996 ) assume that this test statistic has the standard normal distribution under the null hypothesis.”
BackgroundMethodThe consequence of ignoring a nested factor on measures of effect size in analysis of variance.
Wampold, B. E., & Serlin, R. C. (2000) · Psychological Methods · doi:10.1037//1082-989x.5.4.425 · Cited by 186
Design Issues for Experiments in Multilevel Populations
Moerbeek, M., van Breukelen, G. J. P., & Berger, M. P. F. (2000) · Journal of Educational and Behavioral Statistics · doi:10.2307/1165206 · Cited by 156
Self-citationSiddiqui, O., Hedeker, D., Flay, B. R., & Hu, F. B. (1996) · American Journal of Epidemiology · doi:10.1093/oxfordjournals.aje.a008945 · Cited by 152
Moerbeek, M. (2003) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(03)00007-6 · Cited by 152 · cited 1×
“Moerbeek, van Breukelen, and Berger (2003) show that ignoring the nested data structure may lead too an under- or overestimation of the standard error of the treatment effect estimator in an intervention study and consequently inflated type I or type II error rates, respectively.”
BackgroundSelf-citationRestricted unbiased iterative generalized least-squares estimation
GOLDSTEIN, H. (1989) · Biometrika · doi:10.1093/biomet/76.3.622 · Cited by 139 · cited 1×
“This problem may be overcome by using Restricted Iterative Generalized Least Squares (Goldstein, 1989).”
MethodAusems, M., Mesters, I., van Breukelen, G., & de Vries, H. (2002) · Preventive Medicine · doi:10.1006/pmed.2002.1021 · Cited by 103
Sample Size Estimation in Cluster Randomized Studies with Varying Cluster Size
Manatunga, A. K., Hudgens, M. G., & Chen, S. (2001) · Biometrical Journal · doi:10.1002/1521-4036(200102)43:1<75::aid-bimj75>3.0.co;2-n · Cited by 95 · cited 1×
“... design was assumed in the previous sections since that leads to relatively simple formulae for the change in variance components (Equations 14 and 15) when ignoring a level of nesting and to relatively simple formulae for the 1 var( ) in a three or two level design (Tables 1 and 3). Moreover, a balanced design lea…”
BackgroundMarie-Christine, O., & Damme, J. V. (2000) · School Effectiveness and School Improvement · doi:10.1076/0924-3453(200003)11:1;1-#;ft103 · Cited by 90
MULTILEVEL MODELLING NEWSLETTER
Goldstein, H., & Browne, W. J. (2002) · Cited by 71
OPTIMAL EXPERIMENTAL DESIGNS FOR MULTILEVEL MODELS WITH COVARIATES
Moerbeek, M., Van Breukelen, G. J. P., & Berger, M. P. F. (2001) · Communications in Statistics - Theory and Methods · doi:10.1081/sta-100108453 · Cited by 44 · cited 1×
“The variance of the slope estimator, 1 depends on the variance components at and below the level at which the predictor variable varies ( Moerbeek, van Breukelen, & Berger, 2000, 2001; Snijders, 2001).”
BackgroundSelf-citationCarvajal, S. C., Baumler, E. R., Harrist, R. B., & Parcel, G. S. (2001) · Multivariate Behavioral Research · doi:10.1207/s15327906mbr3602_03 · Cited by 32 · cited 1×
“In many studies data have a nested structure, which means that individuals are nested within clusters, which may themselves be nested within higher order clusters, and so on. Examples of such studies are schoolbased smoking prevention or HIV/STD and pregnancy prevention interventions with pupils nested within schools…”
Background
Cited by
Fixed effects and variance components estimation in three-level meta-analysis
Konstantopoulos, S. (2011) · Research Synthesis Methods · doi:10.1002/jrsm.35 · Cited by 521
Detecting Selection Bias in Meta-Analyses with Multiple Outcomes: A Simulation Study
Fernández-Castilla, B., Declercq, L., Jamshidi, L., Beretvas, S. N., Onghena, P., & Van den Noortgate, W. (2021) · The Journal of Experimental Education · doi:10.1080/00220973.2019.1582470 · Cited by 285
Patall, E. A., Steingut, R. R., Vasquez, A. C., Trimble, S. S., Pituch, K. A., & Freeman, J. L. (2018) · Journal of Educational Psychology · doi:10.1037/edu0000214 · Cited by 251
Nagengast, B., & Marsh, H. W. (2012) · Journal of Educational Psychology · doi:10.1037/a0027697 · Cited by 239
Hemming, K., Lilford, R., & Girling, A. J. (2015) · Statistics in Medicine · doi:10.1002/sim.6325 · Cited by 236
“We also note, however, that in parallel CRTs with multiple layers of clustering, a conservative approach to estimate the power is to treat the groups within clusters as one large cluster [16].”
Metadata
- Authors
- Moerbeek, M.
- Venue
- Multivariate Behavioral Research
- Publisher
- Informa UK Limited
- Published
- 2004-01
- Volume
- 39
- Issue
- 1
- Pages
- 129-149
- ISSN
- 0027-3171 (ISSN-L)
- 1532-7906
- OpenAlex
- W1966986302
- Cited by
- 220
- References
- 28
- FWCI
- 4.59
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.