Estimating Multilevel Logistic Regression Models When the Number of Clusters is Low: A Comparison of Different Statistical Software Procedures
Open accessAbstract
Multilevel logistic regression models are increasingly being used to analyze clustered data in medical, public health, epidemiological, and educational research. Procedures for estimating the parameters of such models are available in many statistical software packages. There is currently little evidence on the minimum number of clusters necessary to reliably fit multilevel regression models. We conducted a Monte Carlo study to compare the performance of different statistical software procedures for estimating multilevel logistic regression models when the number of clusters was low. We examined procedures available in BUGS, HLM, R, SAS, and Stata. We found that there were qualitative differences in the performance of different software procedures for estimating multilevel logistic models when the number of clusters was low. Among the likelihood-based procedures, estimation methods based on adaptive Gauss-Hermite approximations to the likelihood (glmer in R and xtlogit in Stata) or adaptive Gaussian quadrature (Proc NLMIXED in SAS) tended to have superior performance for estimating variance components when the number of clusters was small, compared to software procedures based on penalized quasi-likelihood. However, only Bayesian estimation with BUGS allowed for accurate estimation of variance components when there were fewer than 10 clusters. For all statistical software procedures, estimation of variance components tended to be poor when there were only five subjects per cluster, regardless of the number of clusters.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Because validation studies of a certain prediction model are often sparse, traditional (frequentist) meta-analysis models may suffer from convergence issues and yield unreliable estimates of precision and between-study heterogeneity.
“Finally, because validation studies of a certain prediction model are often sparse, traditional (frequentist) meta-analysis models may suffer from convergence issues and yield unreliable estimates of precision and between-study heterogeneity.64”
From A framework for meta-analysis of prediction model studies with binary and time-to-event outcomes (2019) · cites this paper as [64]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
R: A Language and Environment for Statistical Computing
R Core Team (2000) · doi:10.32614/r.manuals · Cited by 355,706 · see citing works
Hierarchical Linear Models: Applications and Data Analysis Methods
Davidian, M. (2003) · Journal of the American Statistical Association · doi:10.1198/jasa.2003.s288 · Cited by 13,820 · see citing works
Multilevel analysis : an introduction to basic and advanced multilevel modeling
Snijders, T. A. B., & Bosker, R. (1999) · Cited by 6,830 · see citing works
Approximate Inference in Generalized Linear Mixed Models
Breslow, N. E., & Clayton, D. G. (1993) · Journal of the American Statistical Association · doi:10.1080/01621459.1993.10594284 · Cited by 4,595 · see citing works
Courgeau, D., & Goldstein, H. (1997) · Population · doi:10.2307/1534624 · Cited by 2,590 · see citing works
Approximate Inference in Generalized Linear Mixed Models
(1993) · doi:10.2307/2290687 · Cited by 1,800 · see citing works
Generalized linear mixed models a pseudo-likelihood approach
Wolfinger, R. D., & O'Connell, M. (1993) · Journal of Statistical Computation and Simulation · doi:10.1080/00949659308811554 · Cited by 1,257 · see citing works
Estimation in Generalized Linear Models with Random Effects
Schall, R. (1991) · Biometrika · doi:10.1093/biomet/78.4.719 · Cited by 1,230 · see citing works
Approximations to the Log-Likelihood Function in the Nonlinear Mixed-Effects Model
Pinheiro, J. C., Bates, D. M., & Pinheiro, J. C. (1995) · Journal of Computational and Graphical Statistics · doi:10.1080/10618600.1995.10474663 · Cited by 1,171 · see citing works
A Language and Program for Complex Bayesian Modelling
Gilks, W., Thomas, A., & Spiegelhalter, D. (1994) · Journal of the Royal Statistical Society Series D (The Statistician) · doi:10.2307/2348941 · Cited by 760 · see citing works
Introducing Markov chain Monte Carlo
Gilks, W., Richardson, S., & Spiegelhalter, D. (1995) · Chapman & Hall/CRC eBooks · doi:10.1201/b14835-6 · Cited by 671 · see citing works
A comparison of Bayesian and likelihood-based methods for fitting multilevel models
Browne, W. J., & Draper, D. (2006) · Bayesian Analysis · doi:10.1214/06-ba117 · Cited by 603 · see citing works
An Assessment of Estimation Procedures for Multilevel Models with Binary Responses
Rodrı́guez, G., & Goldman, N. (1995) · Journal of the Royal Statistical Society Series A (Statistics in Society) · doi:10.2307/2983404 · Cited by 472 · see citing works
Approximations to the Log-Likelihood Function in the Nonlinear Mixed-Effects Model
Pinheiro, J. C., Bates, D. M., & Pinheiro, J. C. (1995) · Journal of Computational and Graphical Statistics · doi:10.2307/1390625 · Cited by 444 · see citing works
A note on Gauss—Hermite quadrature
LIU, Q., & Pierce, D. A. (1994) · Biometrika · doi:10.1093/biomet/81.3.624 · Cited by 350 · see citing works
Estimation in Generalized Linear Models with Random Effects
Schall, R. (1991) · Biometrika · doi:10.2307/2336923 · Cited by 203 · see citing works
Austin, P. C., Manca, A., Zwarenstein, M., Juurlink, D. N., & Stanbrook, M. B. (2009) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2009.06.002 · Cited by 158 · see citing works
Self-citationAn Introduction to Multilevel Regression Models
Austin, P. C., Goel, V., & van Walraven, C. (2001) · Canadian Journal of Public Health · doi:10.1007/bf03404950 · Cited by 146 · see citing works
Self-citationIntroduction to Multilevel Modeling
(2020) · Sage eBooks · doi:10.4135/9781544319315.n1 · Cited by 5 · see citing works
Flagged first, then by citations
Research works printed but not linked: 3
- Bernardo JM, Berger JO, Dawid AP, Smith AFM), pp. 169-193. Oxford: Clarendon Press.
- Raudenbush SW (2008). Many small groups. In: Handbook of Multilevel Analysis (editors: de Leeuw J, Meijer E), pp. 207-236. New York, NY: Springer.
- The International Journal of Biostatistics, Vol. 6 [2010], Iss. 1, Art. 16 DOI: 10.2202/1557-4679.1195
Cited by
Sorted by citations
Multilevel Modelling of Country Effects: A Cautionary Tale
Bryan, M. L., & Jenkins, S. P. (2015) · European Sociological Review · doi:10.1093/esr/jcv059 · Cited by 874 · see citing works
The Effect of Small Sample Size on Two-Level Model Estimates: A Review and Illustration
McNeish, D., & Stapleton, L. M. (2014) · Educational Psychology Review · doi:10.1007/s10648-014-9287-x · Cited by 726 · see citing works
Burke, D. L., Ensor, J., & Riley, R. D. (2017) · Statistics in Medicine · doi:10.1002/sim.7141 · Cited by 538 · see citing works
A framework for meta-analysis of prediction model studies with binary and time-to-event outcomes
Debray, T. P., Damen, J. A., Riley, R. D., Snell, K., Reitsma, J. B., Hooft, L., Collins, G. S., & Moons, K. G. (2019) · Statistical Methods in Medical Research · doi:10.1177/0962280218785504 · Cited by 278 · see citing works
“Finally, because validation studies of a certain prediction model are often sparse, traditional (frequentist) meta-analysis models may suffer from convergence issues and yield unreliable estimates of precision and between-study heterogeneity.(64) For this reason, we presented several methods to overcome these 45 50 55…”
MethodModeling psychophysical data at the population-level: The generalized linear mixed model
Moscatelli, A., Mezzetti, M., & Lacquaniti, F. (2012) · Journal of Vision · doi:10.1167/12.11.26 · Cited by 224 · see citing works
Individual Participant Data Meta-Analysis for a Binary Outcome: One-Stage or Two-Stage?
Debray, T. P. A., Moons, K. G. M., Abo-Zaid, G., Koffijberg, H., & Riley, R. D. (2013) · PLoS ONE · doi:10.1371/journal.pone.0060650 · Cited by 201 · see citing works
Jolani, S., Debray, T. P. A., Koffijberg, H., van Buuren, S., & Moons, K. G. M. (2015) · Statistics in Medicine · doi:10.1002/sim.6451 · Cited by 192 · see citing works
Kerr, J., Sallis, J. F., Owen, N. G., Bourdeaudhuij, I. D., Cerin, E., Sugiyama, T., Reis, R. S., Sarmiento, O. L., Frömel, K., Mitáš, J., et al. (2013) · Journal of Physical Activity and Health · doi:10.1123/jpah.10.4.581 · Cited by 172 · see citing works
Stepped wedge cluster randomised trials: a review of the statistical methodology used and available
Barker, D., McElduff, P., D’Este, C. A., & Campbell, M. J. (2016) · BMC Medical Research Methodology · doi:10.1186/s12874-016-0176-5 · Cited by 160 · see citing works
Hoyle, R. H., & Gottfredson, N. C. (2014) · Prevention Science · doi:10.1007/s11121-014-0489-8 · Cited by 156 · see citing works
Schüle, S. A., & Bolte, G. (2015) · PLoS ONE · doi:10.1371/journal.pone.0123456 · Cited by 148 · see citing works
Visher, C. A., Lattimore, P. K., Barrick, K., & Tueller, S. (2016) · Justice Quarterly · doi:10.1080/07418825.2015.1115539 · Cited by 122 · see citing works
Zant, A. B. V., & Berger, J. A. (2019) · Journal of Personality and Social Psychology · doi:10.1037/pspi0000193 · Cited by 120 · see citing works
Lavigne, É., Bélair, M., Minh, T., Stieb, D. M., Hystad, P., van Donkelaar, A., Martin, R. V., Crouse, D. L., Crighton, E., Chen, H., et al. (2017) · Environment International · doi:10.1016/j.envint.2017.01.004 · Cited by 118 · see citing works
Fusco, F. M., Pisapia, R., Nardiello, S., Cicala, S. D., Gaeta, G. B., & Brancaccio, G. (2019) · BMC Infectious Diseases · doi:10.1186/s12879-019-4285-8 · Cited by 117 · see citing works
Casals, M., Farrés, M. G., & Carrasco, J. L. (2014) · PLoS ONE · doi:10.1371/journal.pone.0112653 · Cited by 115 · see citing works
Oberfeld, D., & Franke, T. (2012) · Behavior Research Methods · doi:10.3758/s13428-012-0281-2 · Cited by 115 · see citing works
Carr, E., & Chung, H. (2014) · Journal of European Social Policy · doi:10.1177/0958928714538219 · Cited by 113 · see citing works
What Drives National Differences in Intensive Grandparental Childcare in Europe?
Gessa, G. D., Glaser, K., Price, D., Ribe, E., & Tinker, A. (2015) · The Journals of Gerontology Series B · doi:10.1093/geronb/gbv007
Astivia, O. L. O., Gadermann, A., & Guhn, M. (2019) · BMC Medical Research Methodology · doi:10.1186/s12874-019-0742-8 · Cited by 100 · see citing works
GROUP COHESIVENESS, GANG MEMBER PRESTIGE, AND DELINQUENCY AND VIOLENCE IN CHICAGO, 1959–1962
Hughes, L. A. (2013) · Criminology · doi:10.1111/1745-9125.12020 · Cited by 98 · see citing works
Regression Analysis of Country Effects Using Multilevel Data: A Cautionary TalePreprint
Bryan, M. L., & Jenkins, S. P. (2013) · SSRN Electronic Journal · doi:10.2139/ssrn.2322088 · Cited by 98 · see citing works
Gurol‐Urganci, I., Bidwell, P., Sevdalis, N., Silverton, L. I., Novis, V., Freeman, R., Hellyer, A., van der Meulen, J., & Thakar, R. (2020) · BJOG An International Journal of Obstetrics & Gynaecology · doi:10.1111/1471-0528.16396 · Cited by 97 · see citing works
Substantial risks associated with few clusters in cluster randomized and stepped wedge designs
Taljaard, M., Teerenstra, S., Ivers, N. M., & Fergusson, D. A. (2016) · Clinical Trials · doi:10.1177/1740774516634316 · Cited by 96 · see citing works
Davey, C., Hargreaves, J., Thompson, J. A., Copas, A. J., Beard, E., Lewis, J. J., & Fielding, K. (2015) · Trials · doi:10.1186/s13063-015-0838-3 · Cited by 93 · see citing works
Recommendations on the Sample Sizes for Multilevel Latent Class Models
Park, J., & Yu, H. (2017) · Educational and Psychological Measurement · doi:10.1177/0013164417719111 · Cited by 86 · see citing works
The Effect of Gender Policies on Fertility: The Moderating Role of Education and Normative Context
Baizan, P., Arpino, B., & Delclòs, C. E. (2016) · European Journal of Population · doi:10.1007/s10680-015-9356-y · Cited by 85 · see citing works
Kim, T. J., Vonneilich, N., Lüdecke, D., & Knesebeck, O. V. D. (2017) · Social Science & Medicine (1967) · doi:10.1016/j.socscimed.2017.01.044 · Cited by 69 · see citing works
Moen, E. L., Fricano-Kugler, C. J., Luikart, B. W., & O’Malley, A. J. (2016) · PLoS ONE · doi:10.1371/journal.pone.0146721 · Cited by 78 · see citing works
The Impact of Sample Size and Other Factors When Estimating Multilevel Logistic Models
Schoeneberger, J. (2016) · The Journal of Experimental Education · doi:10.1080/00220973.2015.1027805 · Cited by 74 · see citing works
Israel, S. (2016) · International Journal for Equity in Health · doi:10.1186/s12939-016-0335-7 · Cited by 73 · see citing works
A 6.4 Mb Duplication of the α-Synuclein Locus Causing Frontotemporal Dementia and Parkinsonism
Kara, E., Kiely, A. P., Proukakis, C., Giffin, N., Love, S., Hehir, J., Rantell, K. R., Pandraud, A., Hernandez, D. G., Nacheva, E. P., et al. (2014) · JAMA Neurology · doi:10.1001/jamaneurol.2014.994 · Cited by 69 · see citing works
Accommodating Small Sample Sizes in Three-Level Models When the Third Level is Incidental
McNeish, D., & Wentzel, K. R. (2016) · Multivariate Behavioral Research · doi:10.1080/00273171.2016.1262236 · Cited by 66 · see citing works
White, J., Hawkins, J. L., Madden, K., Grant, A., Er, V., Angel, L., Pickles, T., Kelson, M. J., Fletcher, A., Murphy, S., et al. (2017) · Public Health Research · doi:10.3310/phr05070 · Cited by 65 · see citing works
Vanishing twin syndrome among ART singletons and pregnancy outcomes
Magnus, M. C., Ghaderi, S., Morken, N., Magnus, P. M., Romundstad, L. B., Skjærven, R., Wilcox, A. J., & Håberg, S. E. (2017) · Human Reproduction · doi:10.1093/humrep/dex277 · Cited by 64 · see citing works
Robust analysis of stepped wedge trials using cluster‐level summaries within periods
Thompson, J. A., Davey, C., Fielding, K., Hargreaves, J., & Hayes, R. (2018) · Statistics in Medicine · doi:10.1002/sim.7668 · Cited by 62 · see citing works
Townsend, L. D., Musci, R. J., Stuart, E. A., Ruble, A. E., Beaudry, M. B., Schweizer, B. W., Owen, M., Goode, C., Johnson, S. L., Bradshaw, C. P., et al. (2017) · Journal of School Health · doi:10.1111/josh.12527 · Cited by 62 · see citing works
Song, F. (., Montabon, F. L., & Xu, Y. (2018) · International Journal of Production Economics · doi:10.1016/j.ijpe.2018.09.020 · Cited by 61 · see citing works
Holm, J. R., & Lorenz, E. (2015) · Industrial and Corporate Change · doi:10.1093/icc/dtv005 · Cited by 53 · see citing works
Brief Educational Intervention Improves Emergency Medical Services Stroke Recognition
Oostema, J. A., Chassee, T. P., Baer, W., Edberg, A., & Reeves, M. J. (2019) · Stroke · doi:10.1161/strokeaha.118.023885 · Cited by 51 · see citing works
Shackleton, N., Hale, D. R., Bonell, C. P., & Viner, R. (2016) · SSM - Population Health · doi:10.1016/j.ssmph.2016.03.005 · Cited by 50 · see citing works
Kim, H. H., & Chun, J. (2018) · Journal of School Health · doi:10.1111/josh.12630 · Cited by 49 · see citing works
Gender and job-related non-formal training: A comparison of 20 countries
Dämmrich, J., Kosyakova, Y., & Blossfeld, H. (2015) · International Journal of Comparative Sociology · doi:10.1177/0020715215626769 · Cited by 48 · see citing works
Applying mixed‐effects modeling to single‐subject designs: An introduction
DeHart, W. B., & Kaplan, B. A. (2019) · Journal of the Experimental Analysis of Behavior · doi:10.1002/jeab.507 · Cited by 47 · see citing works
Does the Package Matter? A Comparison of Five Common Multilevel Modeling Software Packages
McCoach, D. B., Rifenbark, G. G., Newton, S. D., Li, X., Kooken, J. W., Yomtov, D., Gambino, A. J., & Bellara, A. P. (2018) · Journal of Educational and Behavioral Statistics · doi:10.3102/1076998618776348 · Cited by 45 · see citing works
Belief in and use of traditional Chinese medicine in Shanghai older adults: a cross-sectional study
Xin, B., Mu, S., Tan, T., Yeung, A., Gu, D., & Feng, Q. (2020) · BMC Complementary Medicine and Therapies · doi:10.1186/s12906-020-02910-x · Cited by 45 · see citing works
Ayieko, P., Irimu, G., Ogero, M. O., Mwaniki, P., Malla, L., Julius, T., Chepkirui, M., Mbevi, G., Oliwa, J. N., Agweyu, A., et al. (2019) · Implementation Science · doi:10.1186/s13012-019-0868-4 · Cited by 44 · see citing works
Austin, P. C., Kapral, M. K., Vyas, M. V., Fang, J., & Yu, A. (2024) · Neurology · doi:10.1212/wnl.0000000000209947 · Cited by 46 · see citing works
Self-citationDämmrich, J., & Blossfeld, H. (2016) · Acta Sociologica · doi:10.1177/0001699316675022 · Cited by 42 · see citing works
Schüle, S. A., von Kries, R., Fromme, H., & Bolte, G. (2016) · BMC Obesity · doi:10.1186/s40608-016-0106-4 · Cited by 41 · see citing works
Who Wants to Redistribute? An Analysis of 14 Post‐Soviet Nations
Habibov, N. (2011) · Social Policy and Administration · doi:10.1111/j.1467-9515.2011.00834.x · Cited by 39 · see citing works
Motivational interviewing improves depression outcome in primary care: A cluster randomized trial.
Keeley, R. D., Brody, D. S., Engel, M. D., Burke, B. L., Nordstrom, K., Moralez, E. A., Dickinson, L. M., & Emsermann, C. B. (2016) · Journal of Consulting and Clinical Psychology · doi:10.1037/ccp0000124 · Cited by 38 · see citing works
A General Model of Abstention Under Compulsory Voting
Katz, G., & Levin, I. (2016) · Political Science Research and Methods · doi:10.1017/psrm.2016.49 · Cited by 38 · see citing works
McCREDIE, V. A., Alali, A. S., Scales, D. C., Rubenfeld, G. D., Cuthbertson, B. H., & Nathens, A. B. (2018) · Critical Care Medicine · doi:10.1097/ccm.0000000000003149 · Cited by 37 · see citing works
Habibov, N. (2014) · The Journal of Sociology & Social Welfare · doi:10.15453/0191-5096.3981 · Cited by 35 · see citing works
Li, J., Li, S., Galatsch, M., Siegrist, J., Müller, B. H., Hasselhorn, H. M., & NEXT Study Group (2013) · International Journal of Health Services · doi:10.2190/hs.43.3.i · Cited by 33 · see citing works
Leblanc, J., Hejblum, G., Costagliola, D. G., Durand‐Zaleski, I., Lert, F., de Truchis, P., Verbeke, G. N., Rousseau, A., Piquet, H., Simon, F., et al. (2017) · Annals of Emergency Medicine · doi:10.1016/j.annemergmed.2017.09.011 · Cited by 33 · see citing works
Adequate Sample Sizes for a Three-Level Growth Model
Lee, E., & Hong, S. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.685496 · Cited by 33 · see citing works
Hinckson, E. A., Duncan, S., Oliver, M., Mavoa, S., Cerin, E., Badland, H. M., Stewart, T. I., Ivory, V. C., McPhee, J., & Schofield, G. (2014) · BMJ Open · doi:10.1136/bmjopen-2013-004475 · Cited by 32 · see citing works
Design and analysis of trials with a partially nested design and a binary outcome measure
Roberts, S. A., Batistatou, E., & Roberts, S. A. (2015) · Statistics in Medicine · doi:10.1002/sim.6828 · Cited by 30 · see citing works
Electoral Competitiveness and Turnout in British Elections, 1964–2010
Vowles, J., Katz, G., & Stevens, D. P. (2015) · Political Science Research and Methods · doi:10.1017/psrm.2015.67 · Cited by 28 · see citing works
Hennig, E. I., & Mazzi, D. (2018) · Insects · doi:10.3390/insects9030118 · Cited by 28 · see citing works
The Paradox of “Eyes on the Street”: Pedestrian Density and Fear of Crime in Yaoundé, Cameroon
Tchinda, P. E., & Kim, S.-N. (2020) · Sustainability · doi:10.3390/su12135300 · Cited by 28 · see citing works
Multilevel Models for Ordinal Data
Grilli, L., & Rampichini, C. (2011) · Wiley eBooks · doi:10.1002/9781119961154.ch19 · Cited by 27 · see citing works
Effectiveness of a school-based intervention on physical activity and screen time among adolescents
Ahmed, K. R., Kolbe-Alexander, T. L., & Khan, A. (2021) · Journal of science and medicine in sport · doi:10.1016/j.jsams.2021.10.007 · Cited by 27 · see citing works
Majnarić, L., Bekic, S., Babič, F., Pusztová, L., & Paralič, J. (2020) · Medical Science Monitor · doi:10.12659/msm.924281 · Cited by 20 · see citing works
Stavrou, E., Parry, E., Gooderham, P. N., MORLEY, M. J., & Lazarova, M. B. (2021) · Human Resource Management Journal · doi:10.1111/1748-8583.12357 · Cited by 25 · see citing works
Fire, humans, and climate: modeling distribution dynamics of boreal forest waterbirds
Börger, L., & Nudds, T. D. (2013) · Ecological Applications · doi:10.1890/12-1683.1 · Cited by 24 · see citing works
Bayesian estimation with integrated nested Laplace approximation for binary logit mixed models
Grilli, L., Metelli, S., & Rampichini, C. (2014) · Journal of Statistical Computation and Simulation · doi:10.1080/00949655.2014.935377 · Cited by 24 · see citing works
Carrà, G., Crocamo, C., & Bebbington, P. (2017) · International Gambling Studies · doi:10.1080/14459795.2017.1355405 · Cited by 24 · see citing works
Punishing without rewards? A comprehensive examination of the asymmetry in economic voting
Park, J. Y. (2018) · Electoral Studies · doi:10.1016/j.electstud.2018.10.003 · Cited by 22 · see citing works
Effect of Provider Volume on the Accuracy of Hospital Report Cards
Austin, P. C., & Reeves, M. J. (2014) · Circulation Cardiovascular Quality and Outcomes · doi:10.1161/circoutcomes.113.000685 · Cited by 22 · see citing works
Self-citationEadie, D. R., Stead, M., MacKintosh, A. M., Murray, S., Best, C. S., Pearce, J. R., Tisch, C., van der Sluijs, W., Amos, A., MacGregor, A., et al. (2016) · PLoS ONE · doi:10.1371/journal.pone.0152178 · Cited by 22 · see citing works
Rousselière, D., & Rousselière, S. (2017) · PLoS ONE · doi:10.1371/journal.pone.0183213 · Cited by 22 · see citing works
Johnson, C. F. A., Dougall, N. J., Williams, B., MacGillivray, S. A., Buchanan, A. I., & Hassett, R. (2014) · BMC Family Practice · doi:10.1186/s12875-014-0210-9 · Cited by 22 · see citing works
Heubner, L., Oertel, R., Tiebel, O., Mehlig-Warnecke, N., Beyer‐Westendorf, J., Mirus, M., Roessler, M., Renner, B., & Spieth, P. M. (2023) · Anesthesiology · doi:10.1097/aln.0000000000004787 · Cited by 21 · see citing works
Gorey, K. M., Kanjeekal, S. M., Wright, F. C., Hamm, C. M., Luginaah, I., Bartfay, E., Zou, G., Holowaty, E. J., & Richter, N. L. (2015) · International Journal for Equity in Health · doi:10.1186/s12939-015-0246-z · Cited by 21 · see citing works
Performance measures of the bivariate random effects model for meta-analyses of diagnostic accuracy
Díaz, M. (2014) · Computational Statistics & Data Analysis · doi:10.1016/j.csda.2014.09.021 · Cited by 21 · see citing works
Variation in hospital caesarean section rates for preterm births
Bannister-Tyrrell, M., Patterson, J. A., Ford, J. B., Morris, J. M., Nicholl, M., & Roberts, C. L. (2015) · Australian and New Zealand Journal of Obstetrics and Gynaecology · doi:10.1111/ajo.12351 · Cited by 20 · see citing works
Hull, M. C., Reynolds, P. S., & Nunamaker, E. A. (2022) · PLoS ONE · doi:10.1371/journal.pone.0263192 · Cited by 20 · see citing works
Bayesian network meta‐analysis for cluster randomized trials with binary outcomes
Uhlmann, L., Jensen, K., & Kieser, M. (2016) · Research Synthesis Methods · doi:10.1002/jrsm.1210 · Cited by 19 · see citing works
Benedetti, A., Platt, R. W., & Atherton, J. (2014) · PLoS ONE · doi:10.1371/journal.pone.0084601 · Cited by 19 · see citing works
Social Background and the Demographic Life Course: Cross-National Comparisons
Liefbroer, A. C., & Zoutewelle‐Terovan, M. (2021) · Springer eBooks · doi:10.1007/978-3-030-67345-1 · Cited by 19 · see citing works
Bagashka, T. (2013) · Political Research Quarterly · doi:10.1177/1065912913492584 · Cited by 19 · see citing works
Adisso, É. L., Taljaard, M., Stacey, D., Brière, N., Zomahoun, H. T. V., Durand, P. J., Rivest, L., & Légaré, F. (2022) · JMIR Aging · doi:10.2196/39386 · Cited by 18 · see citing works
Liefbroer, A. C., & Zoutewelle‐Terovan, M. (2021) · Springer eBooks · doi:10.1007/978-3-030-67345-1_6 · Cited by 17 · see citing works
Asosega, K. A., Adebanji, A., Aidoo, E. N., & Owusu‐Dabo, E. (2024) · The Scientific World JOURNAL · doi:10.1155/2024/4658333 · Cited by 17 · see citing works
Stolz, E., Mayerl, H., Rásky, É., & Freidl, W. (2019) · PLoS ONE · doi:10.1371/journal.pone.0213787 · Cited by 17 · see citing works
Bassoli, M., & Monticelli, L. (2017) · Acta Politica · doi:10.1057/s41269-017-0047-z · Cited by 16 · see citing works
Individual and Contextual Determinants of Smoking
Goffette, C., & Jacobs-Colas, A. (2016) · Revue Française de Sociologie · doi:10.3917/rfs.572.0213 · Cited by 16 · see citing works
Kwok, H., & Lewis, R. J. (2011) · Circulation Cardiovascular Quality and Outcomes · doi:10.1161/circoutcomes.111.960724 · Cited by 15 · see citing works
Software Packages for Bayesian Multilevel Modeling
Mai, Y., & Zhang, Z. J. (2018) · Structural Equation Modeling A Multidisciplinary Journal · doi:10.1080/10705511.2018.1431545 · Cited by 14 · see citing works
Poisson Multilevel Models with Small Samples
McNeish, D. (2019) · Multivariate Behavioral Research · doi:10.1080/00273171.2018.1545630 · Cited by 14 · see citing works
Finger, C. (2021) · European Sociological Review · doi:10.1093/esr/jcab055 · Cited by 13 · see citing works
Bekić, S., Babič, F., Pavlišková, V., Paralič, J., Wittlinger, T., & Majnarić, L. T. (2021) · Healthcare · doi:10.3390/healthcare9070891 · Cited by 13 · see citing works
Casals, M., Langohr, K., Carrasco, J. L., & Rönnegård, L. (2015) · RECERCAT (Consorci de Serveis Universitaris de Catalunya) · Cited by 12 · see citing works
Evaluation of the effects of multiple providers in complex surgical interventions
Papachristofi, O., Klein, A. A., & Sharples, L. D. (2016) · Statistics in Medicine · doi:10.1002/sim.7057 · Cited by 12 · see citing works
Chen, Z., Zhu, D., Hu, X.-Y., & Gao, G. (2021) · Quality of Life Research · doi:10.1007/s11136-020-02751-8 · Cited by 12 · see citing works
Hierarchical Logistic Regression Models
Wilson, J. R., & Lorenz, K. A. (2015) · ICSA book series in statistics · doi:10.1007/978-3-319-23805-0_10 · Cited by 11 · see citing works
Residual Normality Assumption and the Estimation of Multiple Membership Random Effects Models
Chen, J., & Leroux, A. J. (2018) · Multivariate Behavioral Research · doi:10.1080/00273171.2018.1533445 · Cited by 11 · see citing works
- Authors
- Austin, P. C.
- Affiliations
- Institute for Clinical Evaluative Sciences (CA)
- Corresponding author
- Austin, Peter C
- ORCID
- Austin, Peter C
- Venue
- The International Journal of Biostatistics
- Publisher
- Walter de Gruyter GmbH
- Published
- 2010-04-22
- Volume
- 6
- Issue
- 1
- Pages
- Article 16-Article 16
- ISSN
- 1557-4679 (ISSN-L)
- PMC
- PMC2949382
- OpenAlex
- W2093789600
- Cited by
- 193
- References
- 18
- Citations by year
- 2026: 7 · 2025: 15 · 2024: 7 · 2023: 15 · 2022: 10 · 2021: 11 · 2020: 11 · 2019: 11 · 2018: 13 · 2017: 15 · 2016: 24 · 2015: 20 · 2014: 13 · 2013: 14 · 2012: 3
- FWCI
- 5.72
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- The International Journal of Biostatistics (publishedVersion)
- Europe PMC (PubMed Central) (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Funders
- Heart and Stroke Foundation of Canada
- Institute for Clinical Evaluative Sciences
- Ontario Ministry of Health and Long-Term Care
- Canadian Institutes of Health Research (86508, MOP86508)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Social Sciences › Economics, Econometrics and Finance › Economics and Econometrics › Spatial and Panel Data Analysis
- Keywords
- multilevel logistic regression, small number of clusters, variance component estimation, statistical software comparison, penalized quasi-likelihood, Bayesian estimation, likelihood estimation, clustered data, Monte Carlo simulation
- MeSH terms
- Bayes Theorem, Data Interpretation, Statistical, Humans, Monte Carlo Method, Software, Models, Statistical, Cluster Analysis, Linear Models, Logistic Models
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
