The DOI matches its Crossref record; 25 of the 25 references it cites were found in a registry or an index.
Students Are Rarely Independent: When, Why, and How to Use Random Effects in Discipline-Based Education Research
Open accessAbstract
Discipline-based education researchers have a natural laboratory—classrooms, programs, colleges, and universities. Studies that administer treatments to multiple sections, in multiple years, or at multiple institutions are particularly compelling for two reasons: first, the sample sizes increase, and second, the implementation of the treatments can be intentionally designed and carefully monitored, potentially negating the need for additional control variables. However, when studies are implemented in this way, the observations on students are not completely independent; rather, students are clustered in sections, terms, years, or other factors. Here, I demonstrate why this clustering can be problematic in regression analysis. Fortunately, nonindependence of sampling can often be accounted for with random effects in multilevel regression models. Using several examples, including an extended example with R code, this paper illustrates why and how to implement random effects in multilevel modeling. It also provides resources to promote implementation of analyses that control for the nonindependence inherent in many quasi-random sampling designs.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“We used hierarchical models to account for the nonindependence of student groups (MGS and non-MGS) from a single classroom (ref. 72 and SI Appendix).”
From Active learning narrows achievement gaps for underrepresented students in undergraduate science, technology, engineering, and math (2020) · cites this paper as [72]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Fitting Linear Mixed-Effects Models Using lme4
Bates, D., Mächler, M., Bolker, B., & Walker, S. (2015) · Journal of Statistical Software · doi:10.18637/jss.v067.i01 · Cited by 86,835
Model selection and multimodel inference: a practical information-theoretic approach.
Guthery, F. S., Burnham, K. P., & Anderson, D. R. (2003) · The Journal of Wildlife Management · doi:10.2307/3802723 · Cited by 41,495
Sample Selection Bias as a Specification Error
Heckman, J. J. (1979) · Econometrica · doi:10.2307/1912352 · Cited by 28,779
Mixed effects models and extensions in ecology with R
Zuur, A. F., Ieno, E. N., Walker, N., Saveliev, A. A., & Smith, G. M. (2009) · Statistics for Biology and Health · doi:10.1007/978-0-387-87458-6 · Cited by 17,963
Data Analysis Using Regression and Multilevel/Hierarchical Models
Gelman, A., & Hill, J. (2006) · doi:10.1017/cbo9780511790942 · Cited by 13,597
Active learning increases student performance in science, engineering, and mathematics
Freeman, S., Eddy, S. L., McDonough, M., Smith, M. K., Okoroafor, N., Jordt, H., & Wenderoth, M. P. (2014) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1319030111 · Cited by 9,787
Generalized linear mixed models: a practical guide for ecology and evolution
Bolker, B. M., Brooks, M. E., Clark, C. J., Geange, S. W., Poulsen, J. R., Stevens, M. H. H., & White, J.-S. S. (2009) · Trends in Ecology & Evolution · doi:10.1016/j.tree.2008.10.008 · Cited by 8,910
The standards for educational and psychological testing.
Eignor, D. R. (2013) · American Psychological Association eBooks · doi:10.1037/14047-013 · Cited by 6,985
Standards for Educational and Psychological Testing
(2007) · Sage eBooks · doi:10.4135/9781412952644.n432 · Cited by 3,137
Discovering statistics using R
(2012) · Choice Reviews Online · doi:10.5860/choice.50-2114 · Cited by 2,548
Flagged first, then by citations
Research works printed but not linked: 2
- R Core Team, 2017, A language and environment for statistical computing
- Rosenthal R., 1975, The volunteer subject
Cited by
Sorted by citations
Items 101–114 of 114
Using cueing from question pairs to engage students in reflective thinking: An exploratory study
Ives, J., & Stang, J. B. (2020) · Physics Education Research Conference (PERC) · doi:10.1119/perc.2019.pr.ives
Dewick, P. R., & Liu, S. (2023) · Quality & Quantity · doi:10.1007/s11135-023-01722-8
How students taking introductory biology experience the chemistry content
Mendez, L., Rivera, A. M., Vasquez, I., Aguilar, A. G., Owens, M. T., & Meaders, C. L. (2024) · Journal of Microbiology and Biology Education · doi:10.1128/jmbe.00111-24 · Cited by 2
Athletes vs. Musicians: Perceptions and Preferences in Workplace Social Programs
Celse, J. (2024) · Managerial and Decision Economics · doi:10.1002/mde.4418
Share, rotate, split: The effects of group work role distributions on student outcomes
Feinleib, J., Dew, M., & Holmes, N. G. (2026) · Physical Review Physics Education Research · doi:10.1103/1n3c-v2fj
Yik, B. J., Raker, J. R., Apkarian, N., Stains, M., Henderson, C., Dancy, M., & Johnson, E. (2022) · Figshare · doi:10.6084/m9.figshare.19159114
Xu, S., & Theobald, E. J. (2026) · CBE—Life Sciences Education · doi:10.1187/cbe.25-03-0053
Self-citationItem Position Impact on Student Performance in General Chemistry Multiple-Choice Exams
Terán, Ó. Á. G., Nayyar, P., & Lewis, S. E. (2026) · Journal of Chemical Education · doi:10.1021/acs.jchemed.5c01572
Nazzal, S. A., Assali, A., & Alkowni, R. (2026) · Journal of Science Education and Technology · doi:10.1007/s10956-026-10308-6
Coscia, K. A., Fader, J., Wang, Y., & Genova, L. A. (2026) · Journal of Chemical Education · doi:10.1021/acs.jchemed.5c01279
Is It All About the Costs? Cultural Distance and Adjustment of Recent German Migrants
Genoni, A., Décieux, J. P., & Murdock, E. (2026) · Journal of Cross-Cultural Psychology · doi:10.1177/00220221261455885
Xie, C., Ye, L., Chen, J., Hong, W., Fang, Z., Guan, F., Yao, J., Xiong, J., Xiao, Y., & Liu, T. (2026) · Frontiers in Psychology · doi:10.3389/fpsyg.2026.1815951
Torres, H. L., Johnson-Ojeda, V., Hardy, M., & Frey, R. F. (2026) · Frontiers in Education · doi:10.3389/feduc.2026.1825440
Metadata
- Authors
- Theobald, E.
- Affiliations
- University of Washington (US)
- Corresponding author
- Theobald, Elli
- ORCID
- Theobald, Elli
- Venue
- CBE—Life Sciences Education
- Published
- 2018-09
- Volume
- 17
- Issue
- 3
- Pages
- rm2
- ISSN
- 1931-7913 (ISSN-L)
- PMC
- PMC6234830
- OpenAlex
- W2888039885
- Cited by
- 114
- References
- 25
- Citations by year
- 2026: 11 · 2025: 13 · 2024: 15 · 2023: 17 · 2022: 16 · 2021: 23 · 2020: 9 · 2019: 9 · 2018: 1
- FWCI
- 8.71
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by-nc-sa
- Locations
- CBE—Life Sciences Education (publishedVersion, cc-by-nc-sa)
- Europe PMC (PubMed Central) (publishedVersion, cc-by-nc-sa)
- PubMed Central (publishedVersion, cc-by-nc-sa)
- PubMed (publishedVersion)
- Funders
- University of Washington
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Physical Sciences › Mathematics › Statistics and Probability › Statistics Education and Methodologies
- Social Sciences › Education › Educational Research and Methods
- Keywords
- random effects, multilevel regression, student clustering, treatment implementation, discipline-based education research
- MeSH terms
- Humans, Random Allocation, Research Design, Science, Students, Cluster Analysis, Multilevel Analysis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
