Logistic random effects regression models: a comparison of statistical packages for binary and ordinal outcomes
Open accessThe DOI matches its Crossref record; 4 of the 4 references it cites were found in a registry or an index.
Abstract
Background: Logistic random effects models are a popular tool to analyze multilevel also called hierarchical data with a binary or ordinal outcome. Here, we aim to compare different statistical software implementations of these models. Methods: We used individual patient data from 8509 patients in 231 centers with moderate and severe Traumatic Brain Injury (TBI) enrolled in eight Randomized Controlled Trials (RCTs) and three observational studies. We fitted logistic random effects regression models with the 5-point Glasgow Outcome Scale (GOS) as outcome, both dichotomized as well as ordinal, with center and/or trial as random effects, and as covariates age, motor score, pupil reactivity or trial. We then compared the implementations of frequentist and Bayesian methods to estimate the fixed and random effects. Frequentist approaches included R (lme4), Stata (GLLAMM), SAS (GLIMMIX and NLMIXED), MLwiN ([R]IGLS) and MIXOR, Bayesian approaches included WinBUGS, MLwiN (MCMC), R package MCMCglmm and SAS experimental procedure MCMC. Three data sets (the full data set and two sub-datasets) were analysed using basically two logistic random effects models with either one random effect for the center or two random effects for center and trial. For the ordinal outcome in the full data set also a proportional odds model with a random center effect was fitted. Results: The packages gave similar parameter estimates for both the fixed and random effects and for the binary (and ordinal) models for the main study and when based on a relatively large number of level-1 (patient level) data compared to the number of level-2 (hospital level) data. However, when based on relatively sparse data set, i.e. when the numbers of level-1 and level-2 data units were about the same, the frequentist and Bayesian approaches showed somewhat different results. The software implementations differ considerably in flexibility, computation time, and usability. There are also differences in the availability of additional tools for model evaluation, such as diagnostic plots. The experimental SAS (version 9.2) procedure MCMC appeared to be inefficient. Conclusions: On relatively large data sets, the different software implementations of logistic random effects regression models produced similar results. Thus, for a large data set there seems to be no explicit preference (of course if there is no preference from a philosophical point of view) for either a frequentist or Bayesian approach (if based on vague priors). The choice for a particular implementation may largely depend on the desired flexibility, and the usability of the package. For small data sets the random effects variances are difficult to estimate. In the frequentist approaches the MLE of this variance was often estimated zero with a standard error that is either zero or could not be determined, while for Bayesian methods the estimates could depend on the chosen "non-informative" prior of the variance parameter. The starting value for the variance parameter may be also critical for the convergence of the Markov chain.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
General methods for monitoring convergence of iterative simulations
Brooks, S. P., & Gelman, A. (1998) · Journal of Computational and Graphical Statistics · doi:10.1080/10618600.1998.10474787 · Cited by 6,045
Gelman, A. (2006) · Bayesian Analysis · doi:10.1214/06-ba117a · Cited by 4,050
MIXOR: a computer program for mixed-effects ordinal regression analysis
Hedeker, D., & Gibbons, R. D. (1996) · Computer Methods and Programs in Biomedicine · doi:10.1016/0169-2607(96)01720-8 · Cited by 408 · cited 1×
“Again the number of quadrature points Q determines the desired accuracy [ 20 ].”
BackgroundIMPACT Database of Traumatic Brain Injury: Design And Description
Marmarou, A., Lu, J., Butcher, I., McHugh, G. S., Mushkudiani, N. A., Murray, G. D., Steyerberg, E. W., & Maas, A. I. R. (2007) · Journal of Neurotrauma · doi:10.1089/neu.2006.0036 · Cited by 236 · cited 1×
“For more details on this study, we refer to Marmarou et al [7], and Maas et al [8].”
MethodSelf-citation
Cited by
Get real in individual participant data (IPD) meta‐analysis: a review of the methodology
Debray, T. P. A., Moons, K. G. M., van Valkenhoef, G., Efthimiou, O., Hummel, N., Groenwold, R. H. H., Reitsma, J. B., & on behalf of the GetReal methods review group (2015) · Research Synthesis Methods · doi:10.1002/jrsm.1160 · Cited by 348
“Unfortunately, maximization of the (log-)likelihood function becomes problematic when there are few studies in the meta-analysis or included studies are small (Crowther et al., 2012; Li et al., 2011; Thompson et al., 2001; Higgins et al., 2001).”
BackgroundKim, J., Lim, S., Min, Y. H., Shin, Y.-W., Lee, B., Sohn, G., Jung, K. H., Lee, J.-H., Son, B. H., Ahn, S. H., et al. (2016) · Journal of Medical Internet Research · doi:10.2196/jmir.5598 · Cited by 85
Piontak, J. R., & Schulman, M. D. (2016) · Journal of School Health · doi:10.1111/josh.12458 · Cited by 16
Metadata
- Authors
- Li, B., Lingsma, H. F., Steyerberg, E. W., & Lesaffre, E.
- Venue
- BMC Medical Research Methodology
- Published
- 2011-05-23
- Volume
- 11
- Issue
- 1
- Pages
- 77-77
- ISSN
- 1471-2288 (ISSN-L)
- OpenAlex
- W2015928959
- Cited by
- 142
- References
- 4
- FWCI
- 8.35
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Logistic random effects regression models: a comparison of statistical packages for binary and ordinal outcomes. https://app.cochat.ai/papers/10.1186/1471-2288-11-77
- The paper
- Li, B., Lingsma, H. F., Steyerberg, E. W., & Lesaffre, E. (2011). Logistic random effects regression models: a comparison of statistical packages for binary and ordinal outcomes. BMC Medical Research Methodology. https://doi.org/10.1186/1471-2288-11-77
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.