The DOI matches its Crossref record; 27 of the 27 references it cites were found in a registry or an index.
The Random Effects in Multilevel Models: Getting Them Wrong and Getting Them Right
Open accessAbstract
Many surveys of respondents from multiple countries or subnational regions have now been fielded on multiple occasions. Social scientists are regularly using multilevel models to analyse the data generated by such surveys, investigating variation across both space and time. We show, however, that such models are usually specified erroneously. They typically omit one or more relevant random effects, thereby ignoring important clustering in the data, which leads to downward biases in the standard errors. These biases occur even if the fixed effects are specified correctly; if the fixed effects are incorrect, erroneous specification of the random effects worsens biases in the coefficients. We illustrate these problems using Monte Carlo simulations and two empirical examples. Our recommendation to researchers fitting multilevel models to comparative longitudinal survey data is to include random effects at all potentially relevant levels, thereby avoiding any mismatch between the random and fixed parts of their models.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Specification Tests in Econometrics
Hausman, J. A. (1978) · Econometrica · doi:10.2307/1913827 · Cited by 18,820
Hierarchical Linear Models: Applications and Data Analysis Methods.
Longford, N. T., Bryk, A. S., & Raudenbush, S. W. (1993) · Contemporary Sociology · doi:10.2307/2075823 · Cited by 18,724
Random effects structure for confirmatory hypothesis testing: Keep it maximal
Barr, D. J., Levy, R., Scheepers, C., & Tily, H. J. (2013) · Journal of Memory and Language · doi:10.1016/j.jml.2012.11.001 · Cited by 10,328
Explaining Fixed Effects: Random Effects Modeling of Time-Series Cross-Sectional and Panel Data
Bell, A., & Jones, K. (2015) · Political Science Research and Methods · doi:10.1017/psrm.2014.7 · Cited by 1,515
Panel Models in Sociological Research: Theory into Practice
Halaby, C. N. (2004) · Annual Review of Sociology · doi:10.1146/annurev.soc.30.012703.110629 · Cited by 946
The Rise of Anti-foreigner Sentiment in European Societies, 1988-2000
Semyonov, M., Raijman, R., & Gorodzeisky, A. (2006) · American Sociological Review · doi:10.1177/000312240607100304 · Cited by 862
Two Multilevel Modeling Techniques for Analyzing Comparative Longitudinal Survey Datasets
Fairbrother, M. (2014) · Political Science Research and Methods · doi:10.1017/psrm.2013.24 · Cited by 374
Self-citationHow Robust Standard Errors Expose Methodological Problems They Do Not Fix, and What to Do About It
King, G., & Roberts, M. E. (2015) · Political Analysis · doi:10.1093/pan/mpu015 · Cited by 299
Even in Sweden: The Effect of Immigration on Support for Welfare State Spending
Eger, M. A. (2010) · European Sociological Review · doi:10.1093/esr/jcp017 · Cited by 292
Schlueter, E., & Davidov, E. (2013) · European Sociological Review · doi:10.1093/esr/jcr054 · Cited by 229
Dinesen, P. T. (2013) · European Sociological Review · doi:10.1093/esr/jcr044 · Cited by 198
Rich People, Poor People, and Environmental Concern: Evidence across Nations and Time
Fairbrother, M. (2013) · European Sociological Review · doi:10.1093/esr/jcs068 · Cited by 196
Self-citationExplaining the Trend in Extreme Right-Wing Voting: Germany 1989-1998
Lubbers, M. (2001) · European Sociological Review · doi:10.1093/esr/17.4.431 · Cited by 192
Schmidt-Catran, A. W. (2016) · Socio-Economic Review · doi:10.1093/ser/mwu030 · Cited by 192
Self-citationReligion as Reassurance? Testing the Insecurity Theory in 26 European Countries
Immerzeel, T., & van Tubergen, F. (2013) · European Sociological Review · doi:10.1093/esr/jcr072 · Cited by 190
Support for Redistribution in Western Europe: Assessing the role of religion
Stegmueller, D., Scheepers, P. L. H., Roßteutscher, S., & de Jong, E. (2011) · European Sociological Review · doi:10.1093/esr/jcr011 · Cited by 172
A Bayesian Multilevel Modeling Approach to Time-Series Cross-Sectional Data
Shor, B., Bafumi, J., Keele, L. J., & Park, D. S. (2007) · Political Analysis · doi:10.1093/pan/mpm006 · Cited by 159
Kalmijn, M. (2009) · European Sociological Review · doi:10.1093/esr/jcp035 · Cited by 152
Fekjær, S. N., & Birkelund, G. E. (2007) · European Sociological Review · doi:10.1093/esr/jcm003 · Cited by 121
Biedinger, N., Becker, B., & Rohling, I. (2007) · European Sociological Review · doi:10.1093/esr/jcn001 · Cited by 91
Meulemann, H. (2004) · European Sociological Review · doi:10.1093/esr/20.1.47 · Cited by 80
Huijts, T., Monden, C. W. S., & Kraaykamp, G. (2009) · European Sociological Review · doi:10.1093/esr/jcp019 · Cited by 69
Shi, Y., Leite, W. L., & Algina, J. J. (2009) · British Journal of Mathematical and Statistical Psychology · doi:10.1348/000711008x398968 · Cited by 65
Grotenhuis, M. T., Scholte, M., de Graaf, N. D., & Pelzer, B. J. (2015) · European Sociological Review · doi:10.1093/esr/jcv069 · Cited by 49
The Effects of Relative Group Size on Occupational Outcomes: Turks and Ex-Yugoslavs in Austria
Kogan, I. (2005) · European Sociological Review · doi:10.1093/esr/jci040 · Cited by 25
Justice Perception in Times of Transition: Trends in Germany, 1991-2006
Gerlitz, J.-Y., Mühleck, K., Scheller, P. R., & Schrenker, M. (2010) · European Sociological Review · doi:10.1093/esr/jcq062 · Cited by 25
Meulemann, H. (2010) · European Sociological Review · doi:10.1093/esr/jcq058 · Cited by 16
Cited by
Fixed and random effects models: making an informed choice
Bell, A., Fairbrother, M., & Jones, K. (2019) · Quality & Quantity · doi:10.1007/s11135-018-0802-x · Cited by 1,085
Self-citationThe paradox of inequality: income inequality and belief in meritocracy go hand in hand
Mijs, J. J. B. (2021) · Socio-Economic Review · doi:10.1093/ser/mwy051 · Cited by 585
Heisig, J. P., & Schaeffer, M. (2019) · European Sociological Review · doi:10.1093/esr/jcy053 · Cited by 552
Adolescents’ Psychosocial Well-Being One Year after the Outbreak of the COVID-19 Pandemic in Norway
von Soest, T., Kozák, M., Rodríguez-Cano, R., Fluit, S., Cortés-García, L., Ulset, V. S., Haghish, E. F., & Bakken, A. (2022) · Nature Human Behaviour · doi:10.1038/s41562-021-01255-w · Cited by 122
Educational Systems and Gender Differences in Reading: A Comparative Multilevel Analysis
van Hek, M., Buchmann, C., & Kraaykamp, G. (2019) · European Sociological Review · doi:10.1093/esr/jcy054 · Cited by 72
Metadata
- Authors
- Schmidt-Catran, A. W., & Fairbrother, M.
- Venue
- European Sociological Review
- Publisher
- Oxford University Press (OUP)
- Published
- 2015-09-17
- Volume
- 32
- Issue
- 1
- Pages
- 23-38
- ISSN
- 0266-7215 (ISSN-L)
- 1468-2672
- OpenAlex
- W2289764905
- Cited by
- 372
- References
- 27
- FWCI
- 21.32
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.