The DOI matches its Crossref record; 48 of the 48 references it cites were found in a registry or an index.
Multilevel Modelling of Country Effects: A Cautionary Tale
Open accessAbstract
Country effects on outcomes for individuals are often analysed using multilevel (hierarchical) models applied to harmonized multi-country data sets such as ESS, EU-SILC, EVS, ISSP, and SHARE. We point out problems with the assessment of country effects that appear not to be widely appreciated, and develop our arguments using Monte Carlo simulation analysis of multilevel linear and logit models. With large sample sizes of individuals within each country but only a small number of countries, analysts can reliably estimate individual-level effects but estimates of parameters summarizing country effects are likely to be unreliable. Multilevel modelling methods are no panacea.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
(2001) · RePEc: Research Papers in Economics · Cited by 27,577
Gelman, A., Carlin, J. B., Stern, H. S., & Rubin, D. B. (2003) · doi:10.1201/9780429258480 · Cited by 27,276
Hierarchical Linear Models: Applications and Data Analysis Methods.
Longford, N. T., Bryk, A. S., & Raudenbush, S. W. (1993) · Contemporary Sociology · doi:10.2307/2075823 · Cited by 18,724
Multilevel analysis : an introduction to basic and advanced multilevel modeling
Snijders, T. A. B., & Bosker, R. (1999) · Cited by 13,786
Multilevel analysis techniques and applications
Hox, J. J., Moerbeek, M., & van de Schoot, R. (2002) · Cited by 6,734
A Practitioner’s Guide to Cluster-Robust Inference
Colin Cameron, A., & Miller, D. L. (2015) · Journal of Human Resources · doi:10.3368/jhr.50.2.317 · Cited by 5,037
Small sample inference for fixed effects from restricted maximum likelihood
Kenward, M. G., & Roger, J. H. (1997) · Biometrics · doi:10.2307/2533558 · Cited by 4,552 · cited 1×
“More well known are the small-sample corrections to SEs with associated sample size adjustments that have been developed for the REML estimator (Kenward and Roger, 1997, 2009), and which are available in SAS’s PROC MIXED and R’s ASREML package.”
Method(2009) · Cited by 4,453
Bootstrap-Based Improvements for Inference With Clustered Errors
Cameron, A. C., Gelbach, J. B., & Miller, D. L. (2008) · Review of Economics and Statistics · doi:10.1162/rest.90.3.414 · Cited by 4,417
Sufficient Sample Sizes for Multilevel Modeling
Maas, C. J. M., & Hox, J. J. (2005) · Methodology · doi:10.1027/1614-2241.1.3.86 · Cited by 3,582 · cited 3×
“For example, the model used in the oft-cited Maas and Hox (2005) study included only one individual-level regressor and one country-level regressor (both of which were continuous, normally distributed, variables): see Table 3.”
BackgroundMethodHox, J., Moerbeek, M., & van de Schoot, R. (2010) · doi:10.4324/9780203852279 · Cited by 3,469
Random group effects and the precision of regression estimates
Moulton, B. R. (1986) · Journal of Econometrics · doi:10.1016/0304-4076(86)90021-7 · Cited by 1,626
(2000) · Cited by 1,369
Inference with Difference-in-Differences and Other Panel Data
Donald, S. G., & Lang, K. (2007) · Review of Economics and Statistics · doi:10.1162/rest.89.2.221 · Cited by 1,324
Applied Mixed Models in Medicine
Brown, H., & Presco, R. (2000) · Technometrics · doi:10.1080/00401706.2000.10485742 · Cited by 1,262
Greene, W. (2004) · The Econometrics Journal · doi:10.1111/j.1368-423x.2004.00123.x · Cited by 1,203
Models for Discrete Longitudinal Data
(2005) · Springer Series in Statistics · doi:10.1007/0-387-28980-1 · Cited by 1,021
(2010) · Cited by 1,013
Robustness issues in multilevel regression analysis
Maas, C. J. M., & Hox, J. J. (2004) · Statistica Neerlandica · doi:10.1046/j.0039-0402.2003.00252.x · Cited by 837
How Many Countries for Multilevel Modeling? A Comparison of Frequentist and Bayesian Approaches
Stegmueller, D. (2013) · American Journal of Political Science · doi:10.1111/ajps.12001 · Cited by 818
The design of simulation studies in medical statistics
Burton, A., Altman, D. G., Royston, P., & Holder, R. L. (2006) · Statistics in Medicine · doi:10.1002/sim.2673 · Cited by 752
A comparison of Bayesian and likelihood-based methods for fitting multilevel models
Browne, W. J., & Draper, D. (2006) · Bayesian Analysis · doi:10.1214/06-ba117 · Cited by 603
Bayesian Methods: A Social and Behavioral Sciences Approach
Gill, J. (2002) · Chapman & Hall/CRC eBooks · doi:10.1201/9781420057478 · Cited by 579
Applied Mixed Models in Medicine
Brown, H., & Prescott, R. (2006) · Wiley eBooks · doi:10.1002/0470023589 · Cited by 560
Neuhaus, J. M., Kalbfleisch, J. D., & Hauck, W. W. (1991) · International Statistical Review / Revue Internationale de Statistique · doi:10.2307/1403572 · Cited by 543
An improved approximation to the precision of fixed effects from restricted maximum likelihood
Kenward, M. G., & Roger, J. H. (2009) · Computational Statistics & Data Analysis · doi:10.1016/j.csda.2008.12.013 · Cited by 457
Approximations for Standard Errors of Estimators of Fixed and Random Effects in Mixed Linear Models
Kackar, R. N., & Harville, D. A. (1984) · Journal of the American Statistical Association · doi:10.1080/01621459.1984.10477102 · Cited by 445
Handbook of Multilevel Analysis
(2008) · Springer eBooks · doi:10.1007/978-0-387-73186-5 · Cited by 411
A simulation study of sample size for multilevel logistic regression models
Moineddin, R., Matheson, F. I., & Glazier, R. H. (2007) · BMC Medical Research Methodology · doi:10.1186/1471-2288-7-34 · Cited by 410
Microeconometrics Using Stata, Revised Edition
Cameron, A. C., & Trivedi, P. K. (2010) · Cited by 339
Pinheiro, J. C., & Chao, E. C. (2006) · Journal of Computational and Graphical Statistics · doi:10.1198/106186006x96962 · Cited by 278
Influential Cases in Multilevel Modeling: A Methodological Comment
Van der Meer, T., Te Grotenhuis, M., & Pelzer, B. (2010) · American Sociological Review · doi:10.1177/0003122409359166 · Cited by 225
Improved Estimation Procedures for Multilevel Models with Binary Response: A Case-Study
Rodríguez, G., & Goldman, N. (2001) · Journal of the Royal Statistical Society Series A: Statistics in Society · doi:10.1111/1467-985x.00206 · Cited by 212
Implementation and performance issues in the Bayesian and likelihood fitting of multilevel models
Browne, W. J., & Draper, D. (2000) · Computational Statistics · doi:10.1007/s001800000041 · Cited by 199
Austin, P. C. (2010) · The International Journal of Biostatistics · doi:10.2202/1557-4679.1195 · Cited by 193
Bell, B. A., Morgan, G. B., Schoeneberger, J. A., Kromrey, J. D., & Ferron, J. M. (2014) · Methodology · doi:10.1027/1614-2241/a000062 · Cited by 193
Efficient Estimators for Regressing Regression Coefficients
Hanushek, E. A. (1974) · The American Statistician · doi:10.1080/00031305.1974.10479073 · Cited by 184
Hritcu, R.-O.-S. (2014) · RePEc: Research Papers in Economics · Cited by 160
Unbiasedness of two-stage estimation and prediction procedures for mixed linear models
Kackar, R. N., & Harville, D. A. (1981) · Communications in Statistics - Theory and Methods · doi:10.1080/03610928108828108 · Cited by 159
Leonard, T. J. (2001) · Cited by 159
Approximations for Standard Errors of Estimators of Fixed and Random Effect in Mixed Linear Models
Kackar, R. N., & Harville, D. A. (1984) · Journal of the American Statistical Association · doi:10.2307/2288715 · Cited by 138
Mostly Harmless Econometrics: An Empiricist’s Companion
Harris, D., & Skeels, C. L. (2011) · Economic Record · doi:10.1111/j.1475-4932.2011.00742.x · Cited by 130
A Novel Bootstrap Procedure for Assessing the Relationship between Class Size and Achievement
Carpenter, J. R., Goldstein, H., & Rasbash, J. (2003) · Journal of the Royal Statistical Society Series C (Applied Statistics) · doi:10.1111/1467-9876.00415 · Cited by 124
Sample Size and Accuracy of Estimates in Multilevel Models
Paccagnella, O. (2011) · Methodology · doi:10.1027/1614-2241/a000029 · Cited by 103
A two-stage estimator for probit models with structural group effects
Borjas, G. J., & Sueyoshi, G. T. (1994) · Journal of Econometrics · doi:10.1016/0304-4076(94)90062-0 · Cited by 103
Bayesian Analysis in Applications of Hierarchical Models: Issues and Methods
SELTZER, M. H., Wong, W. H., & Bryk, A. S. (1996) · Journal of Educational and Behavioral Statistics · doi:10.2307/1165214 · Cited by 96
EDA for HLM: Visualization when Probabilistic Inference Fails
Bowers, J., & Drake, K. W. (2005) · Political Analysis · doi:10.1093/pan/mpi031 · Cited by 87
Bayesian Multilevel Analysis and MCMC
Draper, D. (2007) · Handbook of Multilevel Analysis · doi:10.1007/978-0-387-73186-5_2 · Cited by 69
Cited by
Risk perceptions of COVID-19 around the world
Dryhurst, S., Schneider, C. R., Kerr, J., Freeman, A. L. J., Recchia, G., van der Bles, A. M., Spiegelhalter, D., & van der Linden, S. (2020) · Journal of Risk Research · doi:10.1080/13669877.2020.1758193 · Cited by 1,809
Fixed and random effects models: making an informed choice
Bell, A., Fairbrother, M., & Jones, K. (2019) · Quality & Quantity · doi:10.1007/s11135-018-0802-x · Cited by 1,085
The paradox of inequality: income inequality and belief in meritocracy go hand in hand
Mijs, J. J. B. (2021) · Socio-Economic Review · doi:10.1093/ser/mwy051 · Cited by 585
Heisig, J. P., & Schaeffer, M. (2019) · European Sociological Review · doi:10.1093/esr/jcy053 · Cited by 552
Populism as a Problem of Social Integration
Gidron, N., & Hall, P. A. (2020) · Comparative Political Studies · doi:10.1177/0010414019879947 · Cited by 487
Metadata
- Authors
- Bryan, M. L., & Jenkins, S. P.
- Venue
- European Sociological Review
- Publisher
- Oxford University Press (OUP)
- Published
- 2015-05-08
- Volume
- 32
- Issue
- 1
- Pages
- 3-22
- ISSN
- 0266-7215 (ISSN-L)
- 1468-2672
- OpenAlex
- W2163423382
- Cited by
- 866
- References
- 48
- FWCI
- 53.73
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.