Standard Errors and Sample Sizes for Two-Level Research
Open accessAbstract
The hierarchical linear model approach to a two-level design is considered, some variables at the lower level having fixed and others having random regression coefficients. An approximation is derived to the covariance matrix of the estimators of the fixed regression coefficients (for variables at the lower and the higher level) under the assumption that the sample sizes at either level are large enough. This covariance matrix is expressed as a function of parameters occurring in the model. If a research planner can make a reasonable guess as to these parameters, this approximation can be used as a guide to the choice of sample sizes at either level.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Post hoc power analyses confirmed sufficient power (>.80) to detect a mediated pathway comprising paths of medium effect sizes and to detect a medium effect size for treatment condition as a moderator of the mediated pathways.
“Post hoc power analyses using the multilevel power analysis program PinT (Snijders & Bosker, 1993) confirmed that we had sufficient power (>.80) to detect a mediated pathway comprising paths that were of medium effect sizes and to detect a medium effect size for treatment condition as a moderator of the mediated pathways.”
From Respiratory and cognitive mediators of treatment change in panic disorder: Evidence for intervention specificity. (2010) · cites this paper as Snijders & Bosker, 1993
The Determination of Optimal Sample Sizes section describes formulas for power analysis in two-level designs, supporting the citation’s methodological context. Our copy does not identify or describe the PinT program (searched: PinT and software). The >.80 power result concerns the citing authors’ own post hoc analyses, not a result attributed to Snijders and Bosker. (LLM, conf=0.76)
Experimental — accuracy not yet measuredBrief CBT is no more effective than usual care when it comes to preventing repetition of self-harm.
“Tyrer et al reported that brief CBT is no more effective than usual care when it comes to preventing repetition of self-harm, 15”
From Cognitive-behavioural intervention for self-harm: Randomised controlled trial (2008) · cites this paper as [15]
The paper’s discussion of standard errors and sample sizes for two-level research (Abstract; Determination of Optimal Sample Sizes) does not address CBT, usual care, or repetition of self-harm. Searches for those terms and for Tyrer found no relevant passage in our copy. (LLM, conf=0.79)
Experimental — accuracy not yet measuredThis study's finding that time-limited CBT decreases self-harm contrasts with the results reported by Tyrer et al.
“Furthermore, this study's findings that time-limited CBT decreases self-harm contrasts with the results reported by Tyrer et al . 15”
From Cognitive-behavioural intervention for self-harm: Randomised controlled trial (2008) · cites this paper as [15]
The paper’s Abstract discusses standard errors and sample sizes for two-level models, not CBT or self-harm. Searches for Tyrer, time-limited CBT, and self-harm found no passage addressing the cited comparison. (LLM, conf=0.78)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Hierarchical Linear Models: Applications and Data Analysis Methods.
Longford, N. T., Bryk, A. S., & Raudenbush, S. W. (1993) · Contemporary Sociology A Journal of Reviews · doi:10.2307/2075823 · Cited by 18,768 · see citing works
Linear Statistical Inference and its Applications
Rao, C. R. (1973) · Wiley series in probability and statistics · doi:10.1002/9780470316436 · Cited by 10,474 · see citing works
Linear Statistical Inference and Its Applications.
Robertson, C., & Rao, C. R. (1975) · Biometrics · doi:10.2307/2529568 · Cited by 3,611 · see citing works
Hierarchical Linear Models: Applications and Data Analysis Methods.
Goldstein, H., Bryk, A. S., & Raudenbush, S. W. (1993) · Journal of the American Statistical Association · doi:10.2307/2290750 · Cited by 3,320 · see citing works
Application of hierarchical linear models to assessing change.
Bryk, A. S., & Raudenbush, S. W. (1987) · Psychological Bulletin · doi:10.1037/0033-2909.101.1.147 · Cited by 1,505 · see citing works
Multilevel Models in Educational and Social Research
Tate, R. L., & Goldstein, H. (1988) · Journal of Educational Statistics · doi:10.2307/1164754 · Cited by 997 · see citing works
Data Analysis Using Stein's Estimator and its Generalizations
Efron, B., & Morris, C. N. (1975) · Journal of the American Statistical Association · doi:10.1080/01621459.1975.10479864 · Cited by 863 · see citing works
A Hierarchical Model for Studying School Effects
Raudenbush, S. W., & Bryk, A. S. (1986) · Sociology of Education · doi:10.2307/2112482 · Cited by 812 · see citing works
Multilevel Mixed Linear Model Analysis Using Iterative Generalized Least Squares
Goldstein, H. (1986) · Biometrika · doi:10.1093/biomet/73.1.43 · Cited by 798 · see citing works
Statistical Modelling Issues in School Effectiveness Studies
Aitkin, M. A., & Longford, N. T. (1986) · Journal of the Royal Statistical Society Series A (General) · doi:10.2307/2981882 · Cited by 789 · see citing works
Statistical Modelling of Data on Teaching Styles
Aitkin, M. A., Anderson, D. A., & Hinde, J. (1981) · Journal of the Royal Statistical Society Series A (General) · doi:10.2307/2981826 · Cited by 381 · see citing works
Longford, N. T. (1987) · Biometrika · doi:10.1093/biomet/74.4.817 · Cited by 359 · see citing works
Educational Applications of Hierarchical Linear Models: A Review
Raudenbush, S. W. (1988) · Journal of Educational Statistics · doi:10.3102/10769986013002085 · Cited by 301 · see citing works
Empirical bayes meta-analysis.
Raudenbush, S. W., & Bryk, A. S. (1985) · Journal of Educational Statistics · doi:10.3102/10769986010002075 · Cited by 299 · see citing works
Random Coefficient Models for Multilevel Analysis
de Leeuw, J., & Kreft, I. G. G. (1986) · Journal of Educational Statistics · doi:10.3102/10769986011001057 · Cited by 248 · see citing works
The analysis of longitudinal, multilevel data
Raudenbush, S. W. (1989) · International Journal of Educational Research · doi:10.1016/0883-0355(89)90024-4 · Cited by 94 · see citing works
Differences between Scottish Education Authorities in their Examination Attainment[1]
Willms, J. D. (1987) · Oxford Review of Education · doi:10.1080/0305498870130208 · Cited by 40 · see citing works
Flagged first, then by citations
Research works printed but not linked: 4
- Aitkin, M., & Longford, N. T. (1986). Statistical modeling issues in school effective- ness studies (with discussion). Journal of the Royal Statistical Society, Series A , 144, 419-461.
- Cochran, W. G. (1977). Sampling techniques (3rd ed.). New York: Wiley.
- Efron, B., & Morris, C. (1975). Data analysis using Stein's estimator and its gener- alizations. Journal of the American Statistical Association, 74, 311-319.
- Kreft, G. G. (1987). Models and methods for the measurement of school effects. Unpublished doctoral dissertation, University of Amsterdam, The Netherlands.
Cited by
Sorted by citations
Applied Longitudinal Data Analysis for Epidemiology: A Practical Guide
Twisk, J. W. R. (2003) · Cited by 1,244 · see citing works
Multilevel Analysis in Public Health Research
Roux, A. V. D. (2000) · Annual Review of Public Health · doi:10.1146/annurev.publhealth.21.1.171 · Cited by 1,224 · see citing works
“Sample size and power calculations for multilevel hypotheses testing are particularly complex (8, 56, 90).”
BackgroundHox, J. J. (1995) · Utrecht University Repository (Utrecht University) · Cited by 1,179 · see citing works
Building Theoretical and Empirical Bridges Across Levels: Multilevel Research in Management
Hitt, M. A., Beamish, P. W., Jackson, S. E., & Mathieu, J. E. (2007) · Academy of Management Journal · doi:10.5465/amj.2007.28166219 · Cited by 1,138 · see citing works
“Although some software programs allow researchers to make a priori estimates of the power of multilevel designs, these applications are far more complex than their single-level counterparts (cf. Raudenbush, 1997; Snijders & Bosker, 1993).”
BackgroundGabriel, A. S., Podsakoff, N. P., Beal, D. J., Scott, B. A., Sonnentag, S., Trougakos, J. P., & Butts, M. M. (2019) · Organizational Research Methods · doi:10.1177/1094428118802626 · Cited by 755 · see citing works
“Huang and Ryan (2011) similarly took what seems to be a post hoc approach, using the Power analysis IN Two-level designs (PINT) program (Snijders & Bosker, 1993; Snijders, Bosker, & Guldemond, 2003) to determine if their two-level model (events [Level 1; n ¼ 998] nested within people [Level 2; n ¼…”
MethodWestfall, J., Kenny, D. A., & Judd, C. M. (2014) · Journal of Experimental Psychology: General · doi:10.1037/xge0000014 · Cited by 928 · see citing works
“…there is an extensive literature in education and applied statistics on nested designs, commonly referred to as hierarchically nested designs, and this literature has considered issues of power and optimal design (Raudenbush, 1997; Raudenbush & Liu, 2000; Snijders, 2001; Snijders & Bosker, 1993).”
BackgroundThe Effect of Small Sample Size on Two-Level Model Estimates: A Review and Illustration
McNeish, D., & Stapleton, L. M. (2014) · Educational Psychology Review · doi:10.1007/s10648-014-9287-x · Cited by 726 · see citing works
“…and Hibbard 2009; Hox et al. 2012; Konstantopoulos 2010; Kreft 1996; Maas, C and Hox 2004; Maas and Hox 2005; McNeish 2014; Meuleman and Billiet 2009; Moineddin, Matheson, and Glazier 2007; Mok 1995; Paccagnella 2011; Scherbaum and Ferreter 2009; Snijders and Bosker 1993; Stegmueller 2013).”
BackgroundBest practice guidance for linear mixed-effects models in psychological science
Meteyard, L., & Davies, R. (2019) · Journal of Memory and Language · doi:10.1016/j.jml.2020.104092 · Cited by 664 · see citing works
“This also means that accurate power estimation for LMMs requires us to estimate or know the variation within and between sampling units, e.g., for trials within subjects (Snijders & Bosker, 1993; Scherbaum & Ferreter, 2009).”
BackgroundThe power of statistical tests in meta-analysis
Hedges, L. V., & Pigott, T. D. (2001) · Psychological Methods · doi:10.1037/1082-989x.6.3.203 · Cited by 716 · see citing works
Nezlek, J. B. (2001) · Personality and Social Psychology Bulletin · doi:10.1177/0146167201277001 · Cited by 676 · see citing works
Scherbaum, C. A., & Ferreter, J. M. (2009) · Organizational Research Methods · doi:10.1177/1094428107308906 · Cited by 702 · see citing works
“Snijders and Bosker (1993, 1999) presented the following formula based on the t test to determine the maximum value of the standard error for a coefficient that will achieve a given level of power for a particular two-tailed level of a and effect size: Standard Error≤ Effect Size Z1−a=2 + Z1−b…”
BackgroundMethodResultInfluentialIntraclass Correlation Values for Planning Group-Randomized Trials in Education
Hedges, L. V., & Hedberg, E. C. (2007) · Educational Evaluation and Policy Analysis · doi:10.3102/0162373707299706 · Cited by 847 · see citing works
“Such programs include Optimal Design (Raudenbush & Liu, 2000) and PinT ( Snijders & Bosker, 1993 ).”
BackgroundMethodInfluentialBaldwin, S. A., Wampold, B. E., & Imel, Z. E. (2007) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.75.6.842 · Cited by 646 · see citing works
“We used the software program PINT (Bosker, Snijders, & Guldemond, 2003; Snijders & Bosker, 1993) to calculate power to detect a within-therapist coefficient of .20 for the WAI and a between-therapist coefficient of .20 for the WAI, given our sample size of 331 patients and 80 therapists.”
MethodGelman, A., Carlin, J. B., Stern, H. S., Dunson, D. B., Vehtari, A., & Rubin, D. B. (2013) · Chapman & Hall/CRC eBooks · doi:10.1201/b16018-21 · Cited by 631 · see citing works
Statistical analysis and optimal design for cluster randomized trials.
Raudenbush, S. W. (1997) · Psychological Methods · doi:10.1037/1082-989x.2.2.173 · Cited by 636 · see citing works
A solution to dependency: using multilevel analysis to accommodate nested data
Aarts, E., Verhage, M., Veenvliet, J. V., Dolan, C. V., & van der Sluis, S. (2014) · Nature Neuroscience · doi:10.1038/nn.3648 · Cited by 631 · see citing works
A Nondegenerate Penalized Likelihood Estimator for Variance Parameters in Multilevel Models
Chung, Y., Rabe‐Hesketh, S., Dorie, V., Gelman, A., & Liu, J. (2013) · Psychometrika · doi:10.1007/s11336-013-9328-2 · Cited by 566 · see citing works
Mathieu, J. E., Aguinis, H., Culpepper, S. A., & Chen, G. (2012) · Journal of Applied Psychology · doi:10.1037/a0028380 · Cited by 546 · see citing works
Multilevel Modeling: When and Why
Hox, J. (1998) · Studies in Classification, Data Analysis, and Knowledge Organization · doi:10.1007/978-3-642-72087-1_17 · Cited by 500 · see citing works
Power and Sample Size in Multilevel Linear Models
Snijders, T. A. B. (2005) · Encyclopedia of Statistics in Behavioral Science · doi:10.1002/0470013192.bsa492 · Cited by 487 · see citing works
Self-citationSkeletal muscle fiber quality in older men and women
Frontera, W. R., Suh, D., Krivickas, L. S., HUGHES, V. A., Goldstein, R., & Roubenoff, R. (2000) · American Journal of Physiology-Cell Physiology · doi:10.1152/ajpcell.2000.279.3.c611 · Cited by 424 · see citing works
The Accuracy of Multilevel Structural Equation Modeling With Pseudobalanced Groups and Small Samples
Hox, J. J., & Maas, C. J. M. (2001) · Structural Equation Modeling A Multidisciplinary Journal · doi:10.1207/s15328007sem0802_1 · Cited by 414 · see citing works
A simulation study of sample size for multilevel logistic regression models
Moineddin, R., Matheson, F. I., & Glazier, R. H. (2007) · BMC Medical Research Methodology · doi:10.1186/1471-2288-7-34 · Cited by 411 · see citing works
Hedeker, D., Gibbons, R. D., & WATERNAUX, C. M. (1999) · Journal of Educational and Behavioral Statistics · doi:10.3102/10769986024001070 · Cited by 410 · see citing works
“For random-effects models, Snijders and Bosker (1993) present approximate formulas for the standard errors of estimated regression coefficients These standard errors can be used for power calculations for explanatory variables (e.g., group effects, or group by time interactions) in random-effects…”
MethodWORK–FAMILY CONFLICT AND EMOTIONS: EFFECTS AT WORK AND AT HOME
Judge, T. A., Ilieș, R., & Scott, B. A. (2006) · Personnel Psychology · doi:10.1111/j.1744-6570.2006.00054.x · Cited by 402 · see citing works
A framework for testing meso‐mediational relationships in Organizational Behavior
Mathieu, J. E., & Taylor, S. R. (2007) · Journal of Organizational Behavior · doi:10.1002/job.436 · Cited by 374 · see citing works
Solomon, D., Battistich, V. A., Watson, M. S., Schaps, E. M., & Lewis, C. (2000) · Social Psychology of Education · doi:10.1023/a:1009609606692 · Cited by 360 · see citing works
RETRACTED: Structural stigma and all-cause mortality in sexual minority populations
Hatzenbuehler, M. L., Bellatorre, A. C., Lee, Y., Finch, B. K., Muennig, P. A., & Fiscella, K. A. (2013) · Social Science & Medicine · doi:10.1016/j.socscimed.2013.06.005 · Cited by 340 · see citing works
RetractedMultilevel Methods for Public Health Research
Subramanian, S. V., Jones, K., & Duncan, C. (2003) · doi:10.1093/acprof:oso/9780195138382.003.0004 · Cited by 336 · see citing works
Science in the Preschool Classroom: A Programmatic Research Agenda to Improve Science Readiness
Greenfield, D., Jirout, J., Dominguez, X., Greenberg, A., Maier, M., & Fuccillo, J. M. (2009) · Early Education and Development · doi:10.1080/10409280802595441 · Cited by 337 · see citing works
“Finally, the number of classrooms included in the HLM analyses was significantly fewer than is advisable, so these analyses were low on statistical power ( Snijders & Bosker, 1993 ).”
Methodvan der Vegt, G. S., Emans, B. J. M., & de Vliert, E. V. (2001) · Personnel Psychology · doi:10.1111/j.1744-6570.2001.tb00085.x · Cited by 326 · see citing works
Multilevel models in international business research
Peterson, M., Arrégle, J., & Martin, X. (2012) · Research Methods in International Business · doi:10.1057/jibs.2011.59 · Cited by 254 · see citing works
The Etiology of the Multilevel Paradigm in Management Research
Mathieu, J. E., & Chen, G. (2010) · Journal of Management · doi:10.1177/0149206310364663 · Cited by 295 · see citing works
“Although there are software programs to aid researchers in a priori estimation of the power of their multilevel designs, their application is far more complex than their single-level parallels (cf. Raudenbush, 1997; Snijders & Bosker, 1993).”
MethodIacobucci, D., & Ostrom, A. (1996) · International Journal of Research in Marketing · doi:10.1016/0167-8116(95)00034-8 · Cited by 285 · see citing works
McNeish, D. (2017) · Multivariate Behavioral Research · doi:10.1080/00273171.2017.1344538 · Cited by 279 · see citing works
“There are someminor differences between these types ofmodels but, for thepurposeof thismanuscript,we consider themodels tobemoreor less interchangeable. important factor for determining whether the model will be afflicted by small sample issues (Snijders & Bosker, 1993).”
BackgroundMyhill, D. A., Jones, S. M., Lines, H., & Watson, A. (2012) · Research Papers in Education · doi:10.1080/02671522.2011.637640 · Cited by 272 · see citing works
The impact of an integrated approach to science and literacy in elementary school classrooms
Cervetti, G. N., Barber, J., Dorph, R., Pearson, P. D., & Goldschmidt, P. (2012) · Journal of Research in Science Teaching · doi:10.1002/tea.21015 · Cited by 258 · see citing works
“In general, the model consists of three levels and allows for a flexible specification of the covariance structure at every level of the analysis (Snijders and Bosker, 1993).”
BackgroundAllen, C. S., Chen, Q., Willson, V. L., & Hughes, J. N. (2009) · Educational Evaluation and Policy Analysis · doi:10.3102/0162373709352239 · Cited by 257 · see citing works
“Because it is not possible to obtain a true R value for a multilevel model in Mplus (or in HLM), we calculated pseudo-R values according to recommendations by Snijders and Bosker (1993). The within-unit variance explained is a measure of how well the independent”
MethodPower analysis for generalized linear mixed models in ecology and evolution
Johnson, P., Barry, S., Ferguson, H. M., & Müller, P. (2014) · Methods in Ecology and Evolution · doi:10.1111/2041-210x.12306 · Cited by 243 · see citing works
“Simulation-based power analysis methods are available for Gaussian GLMMs (Martin et al. 2011), but there is a lack of guidance and soft- ware facilitating power analysis for scenarios with non-Gauss- ian responses and more complex random effect structures.”
Mueller, K., Hattrup, K., Spieß, S.-O., & Lin‐Hi, N. (2012) · Journal of Applied Psychology · doi:10.1037/a0030204 · Cited by 241 · see citing works
The relevance of multilevel statistical methods for identifying causal neighborhood effects
Subramanian, S. V. (2003) · Social Science & Medicine · doi:10.1016/s0277-9536(03)00415-5 · Cited by 231 · see citing works
Vanyperen, N. W., den Berg, A. E. V., & Willering, M. C. (1999) · Journal of Occupational and Organizational Psychology · doi:10.1348/096317999166734 · Cited by 213 · see citing works
“Ten groups is an acceptable number for performing multilevel analysis, particularly because no explanatory variables at the group level are used in this study (e.g. Snijders & Bosker, 1993).”
MethodMultilevel Modeling: A Review of Methodological Issues and Applications
Dedrick, R. F., Ferron, J. M., Hess, M. R., Hogarty, K. Y., Kromrey, J. D., Lang, T. R., Niles, J. D., & Lee, R. S. (2009) · Review of Educational Research · doi:10.3102/0034654308325581 · Cited by 217 · see citing works
“been considered (Raudenbush, 1997; Snijders & Bosker, 1993 ) as well as the impact of clustering”
BackgroundEffects of the Child Development Project on Students' Drug Use and Other Problem Behaviors
Battistich, V. A., Schaps, E. M., Watson, M. S., Solomon, D., & Lewis, C. (2000) · The Journal of Primary Prevention · doi:10.1023/a:1007057414994 · Cited by 209 · see citing works
“Unfortunately, with a sample of only 24 schools, statistical power to detect the status x year interaction effects of interest here is inadequate (Bassiri, 1988; Snijders & Bosker, 1993).”
BackgroundManagers' gender role attitudes: a country institutional profile approach
Parboteeah, K. P., Hoegl, M., & Cullen, J. B. (2008) · Journal of International Business Studies · doi:10.1057/palgrave.jibs.8400384 · Cited by 201 · see citing works
Love Yourself as a Person, Doubt Yourself as a Therapist?
Nissen‐Lie, H. A., Rønnestad, M. H., Høglend, P. A., Havik, O. E., Solbakken, O. A., Stiles, T. C., & Monsen, J. T. (2015) · Clinical Psychology & Psychotherapy · doi:10.1002/cpp.1977 · Cited by 201 · see citing works
Cognitive-behavioural intervention for self-harm: Randomised controlled trial
Slee, N., Garnefski, N., van der Leeden, R., Arensman, E., & Spinhoven, P. (2008) · British Journal of Psychiatry · doi:10.1192/bjp.bp.107.037564 · Cited by 183 · see citing works
A Monte Carlo sample size study: how many countries are needed for accurate multilevel SEM?
Meuleman, B., & Billiet, J. (2008) · Survey research methods · doi:10.18148/srm/2009.v3i1.666 · Cited by 178 · see citing works
“Snijders and Bosker (1993) have studied the optimality of sample designs for multilevel models, taking budgetary restrictions into account.”
BackgroundResultChristopher, M. S., Hunsinger, M., Goerling, L. R. J., Bowen, S., Rogers, B. S., Gross, C. R., Dapolonia, E., & Pruessner, J. C. (2018) · Psychiatry Research · doi:10.1016/j.psychres.2018.03.059 · Cited by 173 · see citing works
“We examined change from pre-to post-training for each self-report outcome using a multi-level modeling (MLM) approach with restricted maximum likelihood estimation, which is appropriate for smaller sample sizes (Maas and Hox, 2005; Snijders and Bosker, 1993).”
Addressing Data Sparseness in Contextual Population Research
Clarke, P. J., & Wheaton, B. (2007) · Sociological Methods & Research · doi:10.1177/0049124106292362 · Cited by 174 · see citing works
Lapierre, L., van Steenbergen, E. F., Peeters, M. C. W., & Kluwer, E. S. (2015) · Journal of Organizational Behavior · doi:10.1002/job.2075 · Cited by 179 · see citing works
Sastry, N., Ghosh‐Dastidar, B., Adams, J., & Pebley, A. R. (2005) · Social Science Research · doi:10.1016/j.ssresearch.2005.08.002 · Cited by 169 · see citing works
“The goal is generally to do this at the lowest feasible cost (Snijders and Bosker, 1993; Cohen, 1998).”
BackgroundInfluentialFoa, E. B., McLean, C. P., Capaldi, S., & Rosenfield, D. (2013) · JAMA · doi:10.1001/jama.2013.282829 · Cited by 194 · see citing works
Ross, S. W., Romer, N., & Horner, R. H. (2012) · Journal of Positive Behavior Interventions · doi:10.1177/1098300711413820 · Cited by 245 · see citing works
“With a sample of only 184 teachers, the problem of overfitting was a concern, and with that in mind, the analysis still may have led to some overfitting and spurious results (Babyak, 2004; Burnham & Anderson, 2002 ), which may be further exaggerated in the multilevel model (Snijders & Bosker, 1993).”
ResultChristopher, M. S., Goerling, R. J., Rogers, B. S., Hunsinger, M., Baron, G., Bergman, A. L., & Zava, D. T. (2016) · Journal of Police and Criminal Psychology · doi:10.1007/s11896-015-9161-x · Cited by 170 · see citing works
“Multilevel modeling with REML allows for more accurate estimates with small sample sizes and missing data that are missing at random or completely at random (Maas and Hox 2005; Snijders and Bosker 1993).”
MethodLane, S. P., & Hennes, E. P. (2017) · Journal of Social and Personal Relationships · doi:10.1177/0265407517710342 · Cited by 164 · see citing works
“…often be found in books specializing in multilevel modeling and longitudinal design (Ahn, Heo, & Zhang, 2015; Fitzmaurice, Laird, & Ware, 2012; Gelman & Hill, 2006; Hox, 2010; Liu & Liang, 1997; Moerbeek & Teerenstra, 2016; Moerbeek, Van Breukelen, & Berger, 2008; Snijders & Bosker, 1993, 1999).”
Design Issues for Experiments in Multilevel Populations
Moerbeek, M., van Breukelen, G., & Berger, M. P. F. (2000) · Journal of Educational and Behavioral Statistics · doi:10.3102/10769986025003271 · Cited by 156 · see citing works
“Snijders and Bosker (1993) derived optimal allocations of units for two level designs with any number of explanatory variables at both levels.”
BackgroundInfluentialExperimental Plans in Factorial Surveys
Dülmer, H. (2007) · Sociological Methods & Research · doi:10.1177/0049124106292367 · Cited by 155 · see citing works
van de Pol, M. (2011) · Methods in Ecology and Evolution · doi:10.1111/j.2041-210x.2011.00160.x · Cited by 152 · see citing works
Shadish, W. R., Kyse, E. N., & Rindskopf, D. M. (2013) · Psychological Methods · doi:10.1037/a0032964 · Cited by 151 · see citing works
“…no t to be di ss em in at ed br oa dl y. in which persons are nested within some aggregate like classes (Berkhof & Snijders, 2001), often in the context of a grouprandomized trial to test treatment effects (Raudenbush, 1997; Raudenbush & Liu, 2001; Snijders & Bosker, 1993; Zhang & Willson, 2006).”
BackgroundAge and sex affect human muscle fibre adaptations to heavy-resistance strength training
Martel, G. F., Roth, S. M., Ivey, F. M., Lemmer, J. T., Tracy, B. L., Hurlbut, D. E., Metter, E. J., Hurley, B. F., & Rogers, M. A. (2006) · Experimental Physiology · doi:10.1113/expphysiol.2005.032771 · Cited by 149 · see citing works
Murayama, K., Usami, S., & Sakaki, M. (2022) · Psychological Methods · doi:10.1037/met0000330 · Cited by 165 · see citing works
“Formula-based power analysis applies analytic formulae to calculate statistical power (e.g., Raudenbush & Liu, 2000; Snijders & Bosker, 1993).”
MethodMohr, J. J., Gelso, C. J., & Hill, C. E. (2005) · Journal of Counseling Psychology · doi:10.1037/0022-0167.52.3.298 · Cited by 146 · see citing works
“Third, we used a procedure developed by Snijders and Bosker (1993) to determine the power of tests for the significance of individual regression coefficients, which is one of the only clearly delineated methods of assessing power in RIR (Snijders & Bosker, 1999).”
MethodRoosa, M. W., Jones, S. L., Tein, J., & Cree, W. (2003) · American Journal of Community Psychology · doi:10.1023/a:1023070519597 · Cited by 145 · see citing works
Lafit, G., Adolf, J., Dejonckheere, E., Myin‐Germeys, I., Viechtbauer, W., & Ceulemans, E. (2021) · Advances in Methods and Practices in Psychological Science · doi:10.1177/2515245920978738 · Cited by 141 · see citing works
“…these formulas, it is possible to estimate the sample size that allows reaching a predetermined value of power (see, e.g., Cohen, 1988; Hedeker et al., 1999; Moerbeek & Maas, 2005; Moerbeek et al., 2000, 2001; Raudenbush, 1997; Raudenbush & Liu, 2001; Snijders & Bosker, 1993; C. Wang et al., 2015).”
BackgroundTherapist predictors of early patient-rated working alliance: a multilevel approach
Nissen-Lie, H. A., Monsen, J. T., & Rønnestad, M. H. (2010) · Psychotherapy Research · doi:10.1080/10503307.2010.497633 · Cited by 144 · see citing works
“Moreover, in both simulations and other analyses (eg, Raudenbush & Liu, 2000 ; Snijders & Bosker, 1993 ), it seems that “for accuracy and high power, a large number of groups appears more important than a large number of individuals per group” (Hox, 2002 , p.”
MethodJouriles, E. N., McDonald, R., Rosenfield, D., Stephens, N. S., Corbitt‐Shindler, D., & Miller, P. C. (2009) · Journal of Consulting and Clinical Psychology · doi:10.1037/a0015994 · Cited by 144 · see citing works
“.05) group difference for a medium effect size, based on 0.8 power for a growth curve analysis with an average of five data points per family (Snijders & Bosker, 1993).”
MethodInfluentialMeuret, A. E., Rosenfield, D., Seidel, A., Bhaskara, L., & Hofmann, S. G. (2010) · Journal of Consulting and Clinical Psychology · doi:10.1037/a0019552 · Cited by 141 · see citing works
Risk of bias: a simulation study of power to detect study-level moderator effects in meta-analysis
Hempel, S., Miles, J. N., Booth, M. J., Wang, Z., Morton, S. C., & Shekelle, P. G. (2013) · Systematic Reviews · doi:10.1186/2046-4053-2-107 · Cited by 137 · see citing works
“This finding is equivalent to the result found in cluster randomized studies, where the number of individuals is less important than the number of clusters [32,33].”
ResultFord, M. T., Wang, Y., Jin, J., & Eisenberger, R. (2017) · Journal of Occupational Health Psychology · doi:10.1037/ocp0000075 · Cited by 138 · see citing works
“A multilevel power analysis was conducted using the PINT program (Snijders & Bosker, 1993).”
MethodNissen-Lie, H. A., Monsen, J. T., Ulleberg, P., & Rønnestad, M. H. (2013) · Psychotherapy Research · doi:10.1080/10503307.2012.735775 · Cited by 129 · see citing works
“…to an underestimation of the standard errors of therapist predictors at level 3, and would probably result in an inflated Type I error rate (the error of rejecting the null hypothesis when it is true) (de Leeuw & Kreft, 1995; Raudenbush & Bryk, 2002; Snijders & Bosker, 1993; Tasca et al., 2009).”
BackgroundKwok, O.-M., West, S., & Green, S. B. (2007) · Multivariate Behavioral Research · doi:10.1080/00273170701540537 · Cited by 135 · see citing works
“To reduce the number of unknown parameters, the within subject random errors were assumed to be normally distributed with variance equal to one (ie, σ 2 = 1), which is the general practice when conducting power analyses under the MLM framework ( Snijders & Bosker, 1993 ; Bosker, Snijders, & Guldemond, 2003 ).”
MethodSullivan, A. L., Klingbeil, D. A., & Norman, E. R. V. (2013) · School Psychology Review · doi:10.1080/02796015.2013.12087493 · Cited by 125 · see citing works
Samples in applied psychology: Over a decade of research in review.
Shen, W., Kiger, T. B., Davies, S. E., Rasch, R. L., Simon, K. M., & Ones, D. S. (2011) · Journal of Applied Psychology · doi:10.1037/a0023322 · Cited by 160 · see citing works
“The key effect size of interest (i.e., Level 1 or Level 2, fixed or random effect) will dictate where to shift resources for a researcher to reach a desired level of power (see Snijders & Bosker, 1993, and Headrick & Zumbo, 2005, for more details).”
BackgroundTheiss, J. A., & Solomon, D. H. (2006) · Human Communication Research · doi:10.1111/j.1468-2958.2006.00284.x · Cited by 126 · see citing works
“Consistent with Solomon and Knobloch’s (2004) previous test of the relational turbulence model, we operationalized intimacy through a composite measure that incorporated indicators of intimacy associated with developmental patterns (cf.”
MethodApplication of the Hierarchical Bootstrap to Multi-Level Data in Neuroscience
Saravanan, V., Berman, G. J., & Sober, S. J. (2019) · doi:10.1101/819334 · Cited by 170 · see citing works
“Given the relationship between the number of points within a cluster to the Design Effect (DEFF; seeDesign Effect inMaterials andMethods), we would expect the false positive rate to increase with the number of neurons per subject in the traditional analysis but not using the other methods [30, 36, 1].”
BackgroundMethodThe Power of the Test for Treatment Effects in Three-Level Cluster Randomized Designs
Konstantopoulos, S. (2008) · Journal of Research on Educational Effectiveness · doi:10.1080/19345740701692522 · Cited by 124 · see citing works
The typification of Hispanics as criminals and support for punitive crime control policies
Welch, K., Payne, A. A., Chiricos, T., & Gertz, M. (2010) · Social Science Research · doi:10.1016/j.ssresearch.2010.09.012 · Cited by 122 · see citing works
The Dynamic Process of Life Satisfaction
Heller, D. A., Watson, D., & Ilieș, R. (2006) · Journal of Personality · doi:10.1111/j.1467-6494.2006.00415.x · Cited by 123 · see citing works
“To explore this issue, we conducted post-hoc power analyses using the Power IN Two-level designs program (PINT) written by Bosker, Snijders, and Guldemond (2003; see also Snijders & Bosker, 1993, 1999 ).”
MethodFassaert, T. J. L., van Dulmen, S., Schellevis, F., & Bensing, J. M. (2007) · Patient Education and Counseling · doi:10.1016/j.pec.2007.06.011 · Cited by 120 · see citing works
“Kreft argues that 150 groups with five observations within each group should be sufficient to obtain a power of 0.90 [ 33 ].”
BackgroundInfluentialApplications of Multilevel Structural Equation Modeling to Cross-Cultural Research
Cheung, M. W. ‐., & Au, K. Y.-F. (2005) · Structural Equation Modeling A Multidisciplinary Journal · doi:10.1207/s15328007sem1204_5 · Cited by 118 · see citing works
“…findings suggest that increasing the group-level sample size is more beneficial than increasing the individual sample size (e.g., Hox & Maas, 2001; Snijders & Bosker, 1993), it is not clear whether the large individual-level sample size in cross-cultural research will help the group-level model…”
BackgroundFainshmidt, S., Nair, A., & Mallon, M. R. (2017) · International Business Review · doi:10.1016/j.ibusrev.2017.04.002 · Cited by 114 · see citing works
Variability in Self-Reported Normal Sleep Across the Adult Age Span
Dillon, H. R., Lichstein, K. L., Dautovich, N. D., Taylor, D. J. F., Riedel, B. W., & Bush, A. J. (2014) · The Journals of Gerontology Series B · doi:10.1093/geronb/gbu035 · Cited by 118 · see citing works
Multilevel Modelling of Hierarchical Data in Developmental Studies
Boyle, M. H., & Willms, J. D. (2001) · Journal of Child Psychology and Psychiatry · doi:10.1111/1469-7610.00706 · Cited by 112 · see citing works
Huang, J. L. W., & Ryan, A. M. (2011) · Personnel Psychology · doi:10.1111/j.1744-6570.2011.01216.x · Cited by 109 · see citing works
“We used the Power analysis IN Two-level designs (PINT) program (Snijders & Bosker, 1993; Snijders, Bosker, & Guldemond, 2003) for the analysis.”
MethodA Comparison of Methods to Test for Mediation in Multisite Experiments
Pituch, K. A., Whittaker, T. A., & Stapleton, L. M. (2005) · Multivariate Behavioral Research · doi:10.1207/s15327906mbr4001_1 · Cited by 110 · see citing works
Impact of small group size on neighbourhood influences in multilevel models
Theall, K. P., Scribner, R. A., Broyles, S. T., Yu, Q., Chotalia, J., Simonsen, N., Schonlau, M., & Carlin, B. P. (2010) · Journal of Epidemiology & Community Health · doi:10.1136/jech.2009.097956 · Cited by 109 · see citing works
Ilies, R., Huth, M., Ryan, A. M., & Dimotakis, N. (2015) · Journal of Educational Psychology · doi:10.1037/edu0000029 · Cited by 132 · see citing works
Multilevel Modeling: Research-Based Lessons for Substantive Researchers
González‐Romá, V., & Hernández, A. (2017) · Annual Review of Organizational Psychology and Organizational Behavior · doi:10.1146/annurev-orgpsych-041015-062407 · Cited by 105 · see citing works
Smits, J. A. J., Zvolensky, M. J., Davis, M. L., Rosenfield, D., Marcus, B. H., Church, T. S., Powers, M. B., Frierson, G. M., Otto, M. W., Hopkins, L. B., et al. (2016) · Psychosomatic Medicine · doi:10.1097/psy.0000000000000264 · Cited by 103 · see citing works
Myers, T. A., Ridolfi, D. R., Crowther, J. H., & Ciesla, J. A. (2012) · Body Image · doi:10.1016/j.bodyim.2012.03.005 · Cited by 95 · see citing works
Hedges, L. V., & Hedberg, E. C. (2013) · Evaluation Review · doi:10.1177/0193841x14529126 · Cited by 92 · see citing works
McNeish, D. (2017) · Structural Equation Modeling A Multidisciplinary Journal · doi:10.1080/10705511.2017.1280797 · Cited by 92 · see citing works
“This range was selected because 10 is generally seen as the minimum number of clusters needed to conduct a multilevel analysis (Snijders & Bosker, 1993), 50 clusters tends to be the sample size at which small sample issues associated with full maximum likelihood begin to dissipate, and 100 is the…”
BackgroundUsing multilevel models for the analysis of event-related potentials
Volpert‐Esmond, H. I., Page‐Gould, E., & Bartholow, B. D. (2021) · International Journal of Psychophysiology · doi:10.1016/j.ijpsycho.2021.02.006 · Cited by 89 · see citing works
“However, when examining fixed effects, others recommend a minimum threshold of 10 (Snijders and Bosker, 1993), some suggest a minimal threshold of 5 when examining fixed effects but 10 – 100 when examining random effects (McNeish and Stapleton, 2016a), and others suggest that having fewer than 5…”
Keefe, F. J., Shelby, R. A., Somers, T. J., Varia, I., Blazing, M. A., Waters, S. J., McKee, D. C., Silva, S. G., She, L., Blumenthal, J. A., et al. (2011) · Pain · doi:10.1016/j.pain.2010.08.040 · Cited by 89 · see citing works
Three-Level Analysis of Single-Case Experimental Data: Empirical Validation
Moeyaert, M., Ugille, M., Ferron, J. M., Beretvas, S. N., & Noortgate, W. V. D. (2013) · The Journal of Experimental Education · doi:10.1080/00220973.2012.745470 · Cited by 88 · see citing works
“This confirms the theoretical expectation and numerical examples of Snijders and Bosker (1993) who state that the top-level units are of fundamental importance for optimizing power.”
ResultStatistical Power Analysis in Psychological Research
Abraham, W. T., & Russell, D. W. (2008) · Social and Personality Psychology Compass · doi:10.1111/j.1751-9004.2007.00052.x · Cited by 87 · see citing works
Optimal Experimental Designs for Multilevel Logistic Models
Moerbeek, M., van Breukelen, G., & Berger, M. P. F. (2001) · Journal of the Royal Statistical Society Series D (The Statistician) · doi:10.1111/1467-9884.00257 · Cited by 86 · see citing works
“For experiments in multilevel populations with continuous responses, optimal sample sizes have been studied analytically by Donner et al. (1981), Hsieh (1988), Shih (1997), Liu and Liang (1997), Snijders and Bosker (1993), Raudenbush (1997) and Moerbeek et al. (1999, 2000).”
BackgroundPay for performance in emerging markets: Insights from China
Du, J., & Choi, J. N. (2009) · Journal of International Business Studies · doi:10.1057/jibs.2009.40 · Cited by 86 · see citing works
Olatunji, B. O., Rosenfield, D., Tart, C. D., Cottraux, J., Powers, M. B., & Smits, J. A. J. (2013) · Journal of Consulting and Clinical Psychology · doi:10.1037/a0031865 · Cited by 85 · see citing works
- Authors
- Snijders, T. A. B., & Bosker, R. J.
- Affiliations
- University of Groningen (NL)
- University of Twente (NL)
- Venue
- Journal of Educational Statistics
- Published
- 1993-09
- Volume
- 18
- Issue
- 3
- Pages
- 237-259
- ISSN
- 0362-9791 (ISSN-L)
- OpenAlex
- W2097507923
- Cited by
- 487
- References
- 17
- Citations by year
- 2026: 11 · 2025: 13 · 2024: 14 · 2023: 14 · 2022: 14 · 2021: 27 · 2020: 26 · 2019: 34 · 2018: 20 · 2017: 28 · 2016: 22 · 2015: 25 · 2014: 24 · 2013: 35 · 2012: 21
- FWCI
- 1.01
- Type
- journal article
- Language
- en
- Open access
- green
- www.jstor.org
- Locations
- Journal of Educational Statistics (publishedVersion)
- University of Twente Research Information (submittedVersion, other-oa)
- RePEc: Research Papers in Economics (submittedVersion)
- University of Groningen research database (University of Groningen / Centre for Information Technology) (submittedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Physical Sciences › Mathematics › Statistics and Probability › Survey Sampling and Estimation Techniques
- Social Sciences › Decision Sciences › Management Science and Operations Research › Optimal Experimental Design Methods
- Keywords
- two-level designs, multilevel analysis, standard errors, sample size determination, covariance matrix approximation, multilevel data
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
