Students Are Rarely Independent: When, Why, and How to Use Random Effects in Discipline-Based Education Research

Theobald, Elli

doi:10.1187/cbe.17-12-0280

Cited by 114Statistical Methods and Bayesian InferenceStatistics Education and MethodologiesEducational Research and Methods

Students Are Rarely Independent: When, Why, and How to Use Random Effects in Discipline-Based Education Research

Open access

Theobald, Elli·2018·CBE—Life Sciences Education

DOI 10.1187/cbe.17-12-0280PMC PMC6234830

Open this paper in CoChat — free account Open PDF

Abstract

Discipline-based education researchers have a natural laboratory—classrooms, programs, colleges, and universities. Studies that administer treatments to multiple sections, in multiple years, or at multiple institutions are particularly compelling for two reasons: first, the sample sizes increase, and second, the implementation of the treatments can be intentionally designed and carefully monitored, potentially negating the need for additional control variables. However, when studies are implemented in this way, the observations on students are not completely independent; rather, students are clustered in sections, terms, years, or other factors. Here, I demonstrate why this clustering can be problematic in regression analysis. Fortunately, nonindependence of sampling can often be accounted for with random effects in multilevel regression models. Using several examples, including an extended example with R code, this paper illustrates why and how to implement random effects in multilevel modeling. It also provides resources to promote implementation of analyses that control for the nonindependence inherent in many quasi-random sampling designs.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Integrity signals

Registry

Crossref · 10.1187/cbe.17-12-0280

Cites

25 of 25 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Hierarchical models were used to account for nonindependence of student groups from a single classroom.
“We used hierarchical models to account for the nonindependence of student groups (MGS and non-MGS) from a single classroom (ref. 72 and SI Appendix).”
From Active learning narrows achievement gaps for underrepresented students in undergraduate science, technology, engineering, and math (2020) · cites this paper as [72]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Fitting Linear Mixed-Effects Models Using <b>lme4</b>
Bates, D., Mächler, M., Bolker, B., & Walker, S. (2015) · Journal of Statistical Software · doi:10.18637/jss.v067.i01 · Cited by 86,835 · see citing works
Model selection and multimodel inference: a practical information-theoretic approach.
Guthery, F. S., Burnham, K. P., & Anderson, D. R. (2003) · The Journal of Wildlife Management · doi:10.2307/3802723 · Cited by 41,495 · see citing works
Sample Selection Bias as a Specification Error
Heckman, J. J. (1979) · Econometrica · doi:10.2307/1912352 · Cited by 28,779 · see citing works
Mixed effects models and extensions in ecology with R
Zuur, A. F., Ieno, E. N., Walker, N., Saveliev, A. A., & Smith, G. M. (2009) · Statistics for Biology and Health · doi:10.1007/978-0-387-87458-6 · Cited by 17,963 · see citing works
Data Analysis Using Regression and Multilevel/Hierarchical Models
Gelman, A., & Hill, J. (2006) · doi:10.1017/cbo9780511790942 · Cited by 13,597 · see citing works
Active learning increases student performance in science, engineering, and mathematics
Freeman, S., Eddy, S. L., McDonough, M., Smith, M. K., Okoroafor, N., Jordt, H., & Wenderoth, M. P. (2014) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1319030111 · Cited by 9,787 · see citing works
Generalized linear mixed models: a practical guide for ecology and evolution
Bolker, B. M., Brooks, M. E., Clark, C. J., Geange, S. W., Poulsen, J. R., Stevens, M. H. H., & White, J.-S. S. (2009) · Trends in Ecology & Evolution · doi:10.1016/j.tree.2008.10.008 · Cited by 8,910 · see citing works
The standards for educational and psychological testing.
Eignor, D. R. (2013) · American Psychological Association eBooks · doi:10.1037/14047-013 · Cited by 6,975 · see citing works
Standards for Educational and Psychological Testing
(2007) · Sage eBooks · doi:10.4135/9781412952644.n432 · Cited by 3,137 · see citing works
Discovering statistics using R
(2012) · Choice Reviews Online · doi:10.5860/choice.50-2114 · Cited by 2,548 · see citing works
Ecological models and data in R
Bolker, B. M. (2008) · doi:10.2307/j.ctvcm4g37 · Cited by 1,297 · see citing works
Intraclass Correlation Values for Planning Group-Randomized Trials in Education
Hedges, L. V., & Hedberg, E. C. (2007) · Educational Evaluation and Policy Analysis · doi:10.3102/0162373707299706 · Cited by 847 · see citing works
Increased Course Structure Improves Performance in Introductory Biology
Freeman, S., Haak, D., & Wenderoth, M. P. (2011) · CBE—Life Sciences Education · doi:10.1187/cbe.10-08-0105 · Cited by 306 · see citing works
Gender Gaps in Achievement and Participation in Multiple Introductory Biology Classrooms
Eddy, S. L., Brownell, S. E., & Wenderoth, M. P. (2014) · CBE—Life Sciences Education · doi:10.1187/cbe.13-10-0204 · Cited by 281 · see citing works
Discipline-Based Education Research
National Research Council (2012) · National Academies Press eBooks · doi:10.17226/13362 · Cited by 244 · see citing works
Selecting the Best Linear Mixed Model Under REML
Gurka, M. J. (2006) · The American Statistician · doi:10.1198/000313006x90396 · Cited by 200 · see citing works
Student perception of group dynamics predicts individual performance: Comfort and equity matter
Theobald, E. J., Eddy, S. L., Grunspan, D. Z., Wiggins, B. L., & Crowe, A. J. (2017) · PLoS ONE · doi:10.1371/journal.pone.0181336 · Cited by 164 · see citing works
Self-citation
Type I Error Rates from Mixed Effects Model Repeated Measures Versus Fixed Effects Anova with Missing Values Imputed Via Last Observation Carried Forward
Mallinckrodt, C. H., Clark, W. S., & David, S. R. (2001) · Drug Information Journal · doi:10.1177/009286150103500418 · Cited by 160 · see citing works
Handbook of Multilevel Analysis
Leckie, G. (2009) · Journal of the Royal Statistical Society Series A (Statistics in Society) · doi:10.1111/j.1467-985x.2009.00595_3.x · Cited by 158 · see citing works
Is It the Intervention or the Students? Using Linear Regression to Control for Student Characteristics in Undergraduate STEM Education Research
Theobald, R. J., & Freeman, S. (2014) · CBE—Life Sciences Education · doi:10.1187/cbe-13-07-0136 · Cited by 142 · see citing works
Identifying Key Features of Effective Active Learning: The Effects of Writing and Peer Discussion
Linton, D., Pangle, W. M., Wyatt, K. H., Powell, K. N., & Sherwood, R. E. (2014) · CBE—Life Sciences Education · doi:10.1187/cbe.13-12-0242 · Cited by 82 · see citing works
The ICAP Active Learning Framework Predicts the Learning Gains Observed in Intensely Active Classroom Experiences
Wiggins, B., Eddy, S. L., Grunspan, D. Z., & Crowe, A. J. (2017) · AERA Open · doi:10.1177/2332858417708567 · Cited by 82 · see citing works
Context Matters: Volunteer Bias, Small Sample Size, and the Value of Comparison Groups in the Assessment of Research-Based Undergraduate Introductory Biology Lab Courses
Brownell, S. E., Kloser, M. J., Fukami, T., & Shavelson, R. J. (2013) · Journal of Microbiology & Biology Education · doi:10.1128/jmbe.v14i2.609 · Cited by 78 · see citing works
Cognitive Difficulty and Format of Exams Predicts Gender and Socioeconomic Gaps in Exam Performance of Students in Introductory Biology Courses
Wright, C. D., Eddy, S. L., Wenderoth, M. P., Abshire, E. T., Blankenbiller, M., & Brownell, S. E. (2016) · CBE—Life Sciences Education · doi:10.1187/cbe.15-12-0246 · Cited by 70 · see citing works
The Comparison of Model Selection Criteria When Selecting Among Competing Hierarchical Linear Models
Whittaker, T. A., & Furlow, C. F. (2009) · Journal of Modern Applied Statistical Methods · doi:10.22237/jmasm/1241136840 · Cited by 41 · see citing works

Flagged first, then by citations

Research works printed but not linked: 1

Rosenthal R. Rosnow R. (1975). The volunteer subject. Wiley: New York.

Sorted by citations

Active learning narrows achievement gaps for underrepresented students in undergraduate science, technology, engineering, and math
Theobald, E. J., Hill, M. J., Tran, E., Agrawal, S., Arroyo, E. N., Behling, S., Chambwe, N., Cintrón, D. L., Cooper, J. D., Dunster, G., et al. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1916903117 · Cited by 1,387 · see citing works
“A course-section random effect accounts for the nonindependence of observations on multiple students or student groups from each section from each course (72).”
Self-citation
Reducing achievement gaps in undergraduate general chemistry could lift underrepresented students into a “hyperpersistent zone”
Harris, R. B., Mack, M. R., Bryant, J. R., Theobald, E. J., & Freeman, S. (2020) · Science Advances · doi:10.1126/sciadv.aaz5687 · Cited by 185 · see citing works
Self-citation
Student Anxiety and Perception of Difficulty Impact Performance and Persistence in Introductory Biology Courses
England, B. J., Brigati, J. R., Schussler, E. E., & Musgrove, M. M. C. (2019) · CBE—Life Sciences Education · doi:10.1187/cbe.17-12-0284 · Cited by 142 · see citing works
BioSkills Guide: Development and National Validation of a Tool for Interpreting the <i>Vision and Change</i> Core Competencies
Clemmons, A. W., Timbrook, J., Herron, J. C., & Crowe, A. J. (2020) · CBE—Life Sciences Education · doi:10.1187/cbe.19-11-0259 · Cited by 122 · see citing works
Energy savings and thermal comfort in a zero energy office building with fans in Singapore
Kent, M. G., Huynh, N. K., Mishra, A. K., Tartarini, F., Lipczyńska, A., Li, J., Sultan, Z., Goh, E., Karunagaran, G., Natarajan, A., et al. (2023) · Building and Environment · doi:10.1016/j.buildenv.2023.110674 · Cited by 95 · see citing works
Improving Academic Performance, Belonging, and Retention through Increasing Structure of an Introductory Biology Course
Wilton, M., Gonzalez-Niño, E., McPartlan, P., Terner, Z., Christoffersen, R., & Rothman, J. (2019) · CBE - Life Sciences Education · doi:10.1187/cbe.18-08-0155 · Cited by 65 · see citing works
Adolescents’ perceived barriers to accessing sexual and reproductive health services in California: a cross-sectional survey
Decker, M. J., Atyam, T. V., Zárate, C. G., Bayer, A. M., Bautista, C., & Saphir, M. (2021) · BMC Health Services Research · doi:10.1186/s12913-021-07278-3 · Cited by 71 · see citing works
Use of the Test of Scientific Literacy Skills Reveals That Fundamental Literacy Is an Important Contributor to Scientific Literacy
Shaffer, J. F., Ferguson, J., & Denaro, K. (2019) · CBE—Life Sciences Education · doi:10.1187/cbe.18-12-0238 · Cited by 63 · see citing works
Smaller Classes Promote Equitable Student Participation in STEM
Ballen, C. J., Aguillon, S. M., Awwad, A., Bjune, A. E., Challou, D., Drake, A. G., Driessen, M. D., Ellozy, A. R., Ferry, V. E., Goldberg, E. E., et al. (2019) · BioScience · doi:10.1093/biosci/biz069 · Cited by 57 · see citing works
Female In-Class Participation and Performance Increase with More Female Peers and/or a Female Instructor in Life Sciences Courses
Bailey, E. G., Greenall, R. F., Baek, D.-W., Morris, C., Nelson, N., Quirante, T. M., Rice, N. S., Rose, S., & Williams, K. R. (2020) · CBE—Life Sciences Education · doi:10.1187/cbe.19-12-0266 · Cited by 56 · see citing works
Undergraduate Student Concerns in Introductory STEM Courses: What They Are, How They Change, and What Influences Them
Meaders, C. L., Lane, A. K., Morozov, A., Shuman, J. K., Toth, E. S., Stains, M., Stetzer, M. R., Vinson, E. L., Couch, B. A., & Smith, M. K. (2020) · Journal for STEM Education Research · doi:10.1007/s41979-020-00031-1 · Cited by 58 · see citing works
Student Learning Outcomes and Attitudes Using Three Methods of Group Formation in a Nonmajors Biology Class
Donovan, D. A., Connell, G. L., & Grunspan, D. Z. (2018) · CBE—Life Sciences Education · doi:10.1187/cbe.17-12-0283 · Cited by 56 · see citing works
Evaluating the impact of malleable factors on percent time lecturing in gateway chemistry, mathematics, and physics courses
Yik, B. J., Raker, J. R., Apkarian, N., Stains, M., Henderson, C., Dancy, M., & Johnson, E. (2022) · International Journal of STEM Education · doi:10.1186/s40594-022-00333-3 · Cited by 54 · see citing works
Beyond linear regression: A reference for analyzing common data types in discipline based education research
Theobald, E. J., Aikens, M. L., Eddy, S. L., & Jordt, H. (2019) · Physical Review Physics Education Research · doi:10.1103/physrevphyseducres.15.020110 · Cited by 49 · see citing works
Self-citation
Mediation Analysis in Discipline-Based Education Research Using Structural Equation Modeling: Beyond “What Works” to Understand How It Works, and for Whom
Ballen, C. J., & Salehi, S. (2021) · Journal of Microbiology and Biology Education · doi:10.1128/jmbe.00108-21 · Cited by 48 · see citing works
Laboratory Courses with Guided-Inquiry Modules Improve Scientific Reasoning and Experimental Design Skills for the Least-Prepared Undergraduate Students
Blumer, L. S., & Beck, C. W. (2019) · CBE—Life Sciences Education · doi:10.1187/cbe.18-08-0152 · Cited by 40 · see citing works
Memakna Disiplin dalam Kehidupan SMK Negeri di Kabupaten Barito Selatan, Kalimantan Tengah
Siburian, L., Mia Amiani, A., & Munthe, Y. (2023) · DIAJAR Jurnal Pendidikan dan Pembelajaran · doi:10.54259/diajar.v2i2.1374 · Cited by 37 · see citing works
Importance of math prerequisites for performance in introductory physics
Burkholder, E., Murillo-Gonzalez, G., & Wieman, C. (2021) · Physical Review Physics Education Research · doi:10.1103/physrevphyseducres.17.010108 · Cited by 25 · see citing works
“What Will I Experience in My College STEM Courses?” An Investigation of Student Predictions about Instructional Practices in Introductory Courses
Meaders, C. L., Toth, E. S., Lane, A. K., Shuman, J. K., Couch, B. A., Stains, M., Stetzer, M. R., Vinson, E. L., & Smith, M. K. (2019) · CBE—Life Sciences Education · doi:10.1187/cbe.19-05-0084 · Cited by 34 · see citing works
Undergraduate STEM Majors on and off the Pre-Med/Health Track: A STEM Identity Perspective
Dou, R., Cian, H., & Suarez, V. E. (2021) · CBE—Life Sciences Education · doi:10.1187/cbe.20-12-0281 · Cited by 35 · see citing works
Impact of broad categorization on statistical results: How underrepresented minority designation can mask the struggles of both Asian American and African American students
Shafer, D., Mahmood, M. S., & Stelzer, T. (2021) · Physical Review Physics Education Research · doi:10.1103/physrevphyseducres.17.010113 · Cited by 33 · see citing works
Putting the individual into reliability: Bayesian testing of homogeneous within-person variance in hierarchical models
Williams, D. R., Martin, S. R., & Rast, P. (2021) · Behavior Research Methods · doi:10.3758/s13428-021-01646-x · Cited by 31 · see citing works
Meta-analysis of Gender Performance Gaps in Undergraduate Natural Science Courses
Odom, S., Boso, H., Bowling, S., Brownell, S. E., Cotner, S. H., Creech, C., Drake, A. G., Eddy, S. L., Fagbodun, S., Hebert, S. L., et al. (2021) · CBE—Life Sciences Education · doi:10.1187/cbe.20-11-0260 · Cited by 31 · see citing works
Application of Machine Learning Techniques to Predict the Occurrence of Distraction-affected Crashes with Phone-Use Data
Ma, C., Peng, Y., Wu, L., Guo, X., Wang, X., & Kong, X. (2021) · Transportation Research Record Journal of the Transportation Research Board · doi:10.1177/03611981211045371 · Cited by 26 · see citing works
Investigating the day-to-day impact of hypoglycaemia in adults with type 1 or type 2 diabetes: design and validation protocol of the Hypo-METRICS application
Søholm, U., Broadley, M. M., Zaremba, N., Divilly, P., Nefs, G., MAHMOUDI, Z., de Galan, B. E., Pedersen‐Bjergaard, U., Brennan, A., Pollard, D. J., et al. (2022) · BMJ Open · doi:10.1136/bmjopen-2021-051651 · Cited by 25 · see citing works
Why Students Struggle in Undergraduate Biology: Sources and Solutions
Tracy, C. B., Driessen, E. P., Beatty, A. E., Lamb, T., Pruett, J. E., Botello, J. D., Brittain, C., Ford, I. C., Josefson, C. C., Klabacka, R. L., et al. (2022) · CBE - Life Sciences Education · doi:10.1187/cbe.21-09-0289 · Cited by 24 · see citing works
Urgensial Filsafat, Kode Etik dan Profesionalisme Guru di Kalimantan Tengah
Nava, N., Riska, M., Liansih, N., Lukas, Gustina, N., & Munte, A. (2023) · SIBERNETIK Jurnal Pendidikan dan Pembelajaran · doi:10.59632/sjpp.v1i1.18 · Cited by 23 · see citing works
The Relationship between Perceptions of Instructional Practices and Student Self-Efficacy In Guided-Inquiry Laboratory Courses
Beck, C. W., & Blumer, L. S. (2021) · CBE—Life Sciences Education · doi:10.1187/cbe.20-04-0076 · Cited by 21 · see citing works
Examination of quantitative methods for analyzing data from concept inventories
Burkholder, E. W., Walsh, C., & Holmes, N. G. (2020) · Physical Review Physics Education Research · doi:10.1103/physrevphyseducres.16.010141 · Cited by 21 · see citing works
Characterizing change in students' self-assessments of understanding when engaged in instructional activities
Tashiro, J., Parga, D., Pollard, J. A., & Talanquer, V. (2021) · Chemistry Education Research and Practice · doi:10.1039/d0rp00255k · Cited by 21 · see citing works
More than a token photo: humanizing scientists enhances student engagement
Costello, R. A., Driessen, E. P., Kjelvik, M. K., Schultheis, E. H., Youngblood, R. M., Zemenick, A. T., Weber, M. G., & Ballen, C. J. (2025) · Proceedings of the Royal Society B Biological Sciences · doi:10.1098/rspb.2024.0879 · Cited by 19 · see citing works
Getting Students Back on Track: Persistent Effects of Flipping Accelerated Organic Chemistry on Student Achievement, Study Strategies, and Perceptions of Instruction
Reimer, L. C., Denaro, K., He, W., & Link, R. D. (2021) · Journal of Chemical Education · doi:10.1021/acs.jchemed.0c00092 · Cited by 19 · see citing works
A Tale of Two Institutions: Analyzing the Impact of Gamified Student Response Systems on Student Anxiety in Two Different Introductory Biology Courses
Adkins-Jablonsky, S. J., Shaffer, J. F., Morris, J., England, B., & Raut, S. A. (2021) · CBE—Life Sciences Education · doi:10.1187/cbe.20-08-0187 · Cited by 19 · see citing works
Metrics and Methods Used To Compare Student Performance Data in Chemistry Education Research Articles
Mack, M. R., Hensen, C., & Barbera, J. (2019) · Journal of Chemical Education · doi:10.1021/acs.jchemed.8b00713 · Cited by 20 · see citing works
Effects of “we”-framing on young children’s commitment, sharing, and helping
Vasil, J., & Tomasello, M. (2021) · Journal of Experimental Child Psychology · doi:10.1016/j.jecp.2021.105278 · Cited by 16 · see citing works
Moving the Needle: Evidence of an Effective Study Strategy Intervention in a Community College Biology Course
Vemu, S., Denaro, K., Sato, B. K., & Williams, A. E. (2022) · CBE—Life Sciences Education · doi:10.1187/cbe.21-08-0216 · Cited by 15 · see citing works
“…performance was fit to the data using linear mixed-effects models to account for the correlation of students nested within a class (Theobald and Freeman, 2014; Theobald, 2018), models which are well studied (Fahrmeir and Tutz, 2001; Diggle et al. , 2002) and developed by Laird and Ware (1982).”
The Case for Biocalculus: Improving Student Understanding of the Utility Value of Mathematics to Biology and Affect toward Mathematics
Aikens, M. L., Eaton, C., & Highlander, H. (2021) · CBE - Life Sciences Education · doi:10.1187/cbe.20-06-0124 · Cited by 14 · see citing works
Effects of Remote Teaching in a Crisis on Equity Gaps and the Constructivist Learning Environment in an Introductory Biology Course Series
Zuckerman, A. L., Hardesty, R. A., Denaro, K., Lo, S. M., & Owens, M. T. (2021) · Journal of Microbiology and Biology Education · doi:10.1128/jmbe.v22i1.2293 · Cited by 15 · see citing works
Psychometric properties of an innovative smartphone application to investigate the daily impact of hypoglycemia in people with type 1 or type 2 diabetes: The Hypo-METRICS app
Søholm, U., Broadley, M. M., Zaremba, N., Divilly, P., Nefs, G., Carlton, J., Mader, J. K., Baumann, P. M., Gomes, M., Martine‐Edith, G., et al. (2023) · PLoS ONE · doi:10.1371/journal.pone.0283148 · Cited by 13 · see citing works
Virtually the Same? Evaluating the Effectiveness of Remote Undergraduate Research Experiences
Hess, R. A., Erickson, O. A., Cole, R. B., Isaacs, J. M., Álvarez-Clare, S., Arnold, J., Augustus‐Wallace, A., Ayoob, J. C., Berkowitz, A. R., Branchaw, J. L., et al. (2023) · CBE—Life Sciences Education · doi:10.1187/cbe.22-01-0001 · Cited by 12 · see citing works
Mindfulness-based stress reduction for elementary school teachers: a randomized controlled trial
Lensen, J. H., Stoltz, S. E. M. J., Kleinjan, M., Kraiss, J. T., Scholte, R. H. J., & Speckens, A. E. M. (2024) · Frontiers in Education · doi:10.3389/feduc.2024.1385375 · Cited by 17 · see citing works
What is the basis of teacher judgment of student cognitive abilities and academic achievement and what affects its accuracy?
Jabůrek, M., Cígler, H., Valešová, T., & Portešová, Š. (2022) · Contemporary Educational Psychology · doi:10.1016/j.cedpsych.2022.102068 · Cited by 12 · see citing works
Exploring the contributions of self-efficacy and test anxiety to gender differences in assessments
Stang, J., Altiere, E., Ives, J., & Dubois, P. (2020) · Physics Education Research Conference Proceedings · doi:10.1119/perc.2020.pr.stang · Cited by 8 · see citing works
Stressors in a bottle: A microcosm study on phytoplankton assemblage response to extreme precipitation events under climate warming
Armstrong, M., Zhan, Q., Munthali, E., Jin, H., Teurlincx, S., Peters, P., Lürling, M., & Domis, L. N. D. S. (2023) · Freshwater Biology · doi:10.1111/fwb.14109 · Cited by 11 · see citing works
Pushing toward systemic change in the Capitalocene: Investigating the efficacy of existing behavior prediction models on individual and collective pro-environmental actions in high school students
Gibson, L., Stevenson, K. T., Busch, K. C., Cutts, B. B., Seekamp, E., & Krementz, S. (2024) · The Journal of Environmental Education · doi:10.1080/00958964.2023.2259853 · Cited by 12 · see citing works
What College Biology Students Know about How Vaccines Work
Kahlon, G., Waheed, F., & Owens, M. T. (2022) · CBE—Life Sciences Education · doi:10.1187/cbe.20-12-0294 · Cited by 11 · see citing works
Gendered Performance Gaps in an Upper-Division Biology Course: Academic, Demographic, Environmental, and Affective Factors
Farrar, V. S., Aguayo, B.-Y. C., & Caporale, N. (2023) · CBE—Life Sciences Education · doi:10.1187/cbe.23-03-0041 · Cited by 13 · see citing works
Tracing Undergraduate Science Learners’ Digital Cognitive Strategy Use and Relation to Performance
Mefferd, K., & Bernacki, M. L. (2023) · Journal of Science Education and Technology · doi:10.1007/s10956-022-10018-9 · Cited by 11 · see citing works
Try Before You Buy: Are There Benefits to a Random Trial Period before Students Choose Their Collaborative Teams?
Samudra, S. P., Walters, C., Williams-Dobosz, D., Shah, A., & Brickman, P. (2023) · CBE—Life Sciences Education · doi:10.1187/cbe.23-01-0011 · Cited by 12 · see citing works
Detecting variability: A study on the application of bayesian multilevel modelling to archaeological data. Evidence from the Neolithic Adriatic and the Bronze Age Aegean
Fernée, C., & Trimmis, K. P. (2021) · Journal of Archaeological Science · doi:10.1016/j.jas.2021.105346 · Cited by 10 · see citing works
Lateral Reading on the Open InternetPreprint
Wineburg, S., Breakstone, J., McGrew, S., Smith, M., & Ortega, T. (2021) · SSRN Electronic Journal · doi:10.2139/ssrn.3936112 · Cited by 10 · see citing works
A survey of study skills of first-year university students: the relationships of strategy to gender, ethnicity and course type
Williams, A. E., Denaro, K., Dennin, M. B., & Sato, B. K. (2021) · Journal of Applied Research in Higher Education · doi:10.1108/jarhe-10-2019-0272 · Cited by 9 · see citing works
Polar knowledge of US students as indicated by an online Kahoot! quiz game
Pfirman, S., Hamilton, L. C., Turrin, M., Narveson, C., & Lloyd, C. A. (2021) · Journal of Geoscience Education · doi:10.1080/10899995.2021.1877526 · Cited by 9 · see citing works
Teilnahme an mathematischen Vorkursen und langfristiger Studienerfolg. Eine empirische Untersuchung
Gerdes, A., Halverscheid, S., & Schneider, S. (2021) · Journal für Mathematik-Didaktik · doi:10.1007/s13138-021-00194-3 · Cited by 9 · see citing works
Active Learning Strategies for Biodiversity Science
Chodkowski, N., O’Grady, P., Specht, C. D., & Zamudio, K. R. (2022) · Frontiers in Education · doi:10.3389/feduc.2022.849300 · Cited by 9 · see citing works
Measuring and correcting staff variability in large-scale OSCEs
Haviari, S., de Tymowski, C., Burnichon, N., Lemogne, C., Flamant, M., Ruszniewski, P. B., Bensaadi, S., Mercier, G., Hamaoui, H., Université Paris Cité OSCE study group, et al. (2024) · BMC Medical Education · doi:10.1186/s12909-024-05803-6 · Cited by 8 · see citing works
Using social influence models to characterize student interest in a general chemistry peer-led team learning setting
McAlpin, J. D., Kulatunga, U., & Lewis, J. E. (2023) · Chemistry Education Research and Practice · doi:10.1039/d2rp00296e · Cited by 8 · see citing works
Metacognitive Exam Preparation Assignments in an Introductory Biology Course Improve Exam Scores for Lower ACT Students Compared with Assignments that Focus on Terms
Angell, D. K., Lane-Getaz, S., Okonek, T., & Smith, S. (2024) · CBE—Life Sciences Education · doi:10.1187/cbe.22-10-0212 · Cited by 8 · see citing works
Optional Exam Retakes Reduce Anxiety but may Exacerbate Score Disparities Between Students with Different Social Identities
Supriya, K., Bang, C., Ebie, J. D., Pagliarulo, C., Tucker, D. B., Villegas, K., Wright, C. D., & Brownell, S. E. (2024) · CBE—Life Sciences Education · doi:10.1187/cbe.21-11-0320 · Cited by 8 · see citing works
Textbook outcomes in heart transplantation: A quality metric for the modern era
Bakhtiyar, S. S., Sakowitz, S., Ali, K., Coaston, T. N., Verma, A., Chervu, N. L., & Benharash, P. (2023) · Surgery · doi:10.1016/j.surg.2023.03.014 · Cited by 8 · see citing works
The retention benefits of cumulative versus non-cumulative midterms in introductory biology may depend on students’ reasoning skills
Bailey, E. G., Greenall, R. F., Tullis, M. M., & Williams, K. R. (2021) · PLoS ONE · doi:10.1371/journal.pone.0250143 · Cited by 7 · see citing works
The contributions of reading and phonological awareness for spelling in grade three isiXhosa learners
Daries, M. A., Bowles, T. N., & Schaefer, M. (2022) · Reading & Writing · doi:10.4102/rw.v13i1.365 · Cited by 7 · see citing works
Window View Satisfaction Assessment Method: A Comparison of Physical Space, Virtual Reality, and Digital Image
Kim, J., Kral, K., Ko, W. H., Kent, M. G., Schiavon, S., & Dogan, T. (2025) · LEUKOS The Journal of the Illuminating Engineering Society of North America · doi:10.1080/15502724.2025.2528893 · Cited by 8 · see citing works
Intersecting Identities: A Look at How Ethnic Identity Interacts With Science Identity in Native Hawaiian and Pacific Islander Students
Greenall, R. F., de Alba, J. G., Nichols, S., Allen, G. E. K., & Bailey, E. G. (2023) · CBE—Life Sciences Education · doi:10.1187/cbe.23-01-0010 · Cited by 7 · see citing works
Journal reflections shed light on challenges students face in an introductory field biology course
Treibergs, K. A., Esparza, D., Yamazaki, J. A., & Smith, M. K. (2023) · Ecosphere · doi:10.1002/ecs2.4509 · Cited by 6 · see citing works
Successful Problem Solving in Genetics Varies Based on Question Content
Avena, J. S., McIntosh, B. B., Whitney, O. N., Wiens, A., & Knight, J. (2021) · CBE—Life Sciences Education · doi:10.1187/cbe.21-01-0016 · Cited by 7 · see citing works
Quantifying ento-literacy: development and validation of an international insect-focused attitude and knowledge survey instrument
Lucky, A., Janštová, V., Novotný, P., & Mourek, J. (2025) · International Journal of STEM Education · doi:10.1186/s40594-025-00532-8 · Cited by 5 · see citing works
Malaria prevalence dynamics and risk covariates among children under 5 in Ghana: insights from a Bayesian multilevel approach
Asosega, K. A., Adebanji, A., Aidoo, E., Owusu-Dabo, E., & Tawiah, K. (2025) · BMJ Open · doi:10.1136/bmjopen-2024-088910 · Cited by 3 · see citing works
The effectiveness of a web-based Dutch parenting program to prevent overweight in children 9–13 years of age: Results of a two-armed cluster randomized controlled trial
Ruiter, E. L. M., Molleman, G. R. M., Kleinjan, M., Kraiss, J. T., ten Klooster, P. M., van der Velden, K., Engels, R. C. M. E., & Fransen, G. A. J. (2022) · PLoS ONE · doi:10.1371/journal.pone.0276168 · Cited by 6 · see citing works
Investigating Student Intrinsic Motivation Trends over One Semester in an Online General Chemistry II Course
Liu, Y., & Kim, E. (2024) · Journal of Chemical Education · doi:10.1021/acs.jchemed.3c00245 · Cited by 5 · see citing works
Understanding how professional development opportunities affect open educational resource sharing
Flowers, S., Hazlett, Z. S., Ramirez, M. L., Treibergs, K. A., Vinson, E. L., Smith, M. K., & Knight, J. (2024) · Frontiers in Education · doi:10.3389/feduc.2024.1422383 · Cited by 5 · see citing works
Tardiness Increases in Winter: Evidence for Annual Rhythms in Humans
Zerbini, G., van der Vinne, V., Otto, L., Monecke, S., Kantermann, T., & Merrow, M. W. (2019) · Journal of Biological Rhythms · doi:10.1177/0748730419876781 · Cited by 5 · see citing works
Verbalized Studying and Elaborative Interrogation in the Virtual Classroom: Students with Social Anxiety Prefer Working Alone, but Working with a Peer Does Not Hurt Their Learning
Novak, R. T., Bailey, E. G., Blinsky, B. D., Soffe, B. W., Patterson, D., Ockey, J., & Jensen, J. L. (2022) · Journal of Microbiology and Biology Education · doi:10.1128/jmbe.00232-21 · Cited by 5 · see citing works
A CURE Lab in Introductory Biology at a Regional Comprehensive University Negatively Impacts Student Success in the Associated Lecture Course Among Students from Groups Underrepresented in Science
Casper, A. M., & Laporte, M. M. (2024) · CBE—Life Sciences Education · doi:10.1187/cbe.23-06-0122 · Cited by 4 · see citing works
Supporting Students’ Self-Regulated Learning in an Introductory Physics Course
Rieger, G. W., McIver, J., Mazabel, S., & Burkholder, E. W. (2022) · The Physics Teacher · doi:10.1119/5.0062172 · Cited by 4 · see citing works
The Use of Physically Active Academic Lessons During the Transition to Face-to-Face Classes
Beserra, V., Nussbaum, M., Navarrete, M., Cabezas, N. G., & Alvares, D. (2024) · SAGE Open · doi:10.1177/21582440241255429 · Cited by 3 · see citing works
An Asynchronous Chemistry-in-biology Intervention Improves Student Content Knowledge and Performance in Introductory Biology
Meaders, C. L., Mendez, L., Aguilar, A. G., Rivera, A. T., Vasquez, I., Mueller, L. O., & Owens, M. T. (2025) · CBE - Life Sciences Education · doi:10.1187/cbe.24-05-0151 · Cited by 4 · see citing works
Student identities predict classmate and instructor fear of negative evaluation among undergraduates in large-enrollment introductory biology courses
Cooper, K. M., Busch, C. A., & Brownell, S. E. (2025) · Journal of Microbiology & Biology Education · doi:10.1128/jmbe.00191-24 · Cited by 3 · see citing works
Strategically creating maximally heterogeneous lab groups did not improve group performance in an introductory biology lab class
Navlekar, A. S., Theobald, E. J., Griffith, K., & Limeri, L. B. (2025) · PLoS ONE · doi:10.1371/journal.pone.0323799 · Cited by 3 · see citing works
Self-citation
Could Instructor Talk Drive CURE Effectiveness? A Comparative Study of Instructor Talk in Introductory Lab Courses
Zajic, C. J., Subramanian, K., Adulla, A., Allen, Z. A., Blitchington, M. B., Brotzman, A., Carrillo, E., Dhruv, J., Evans, T., Haider, S., et al. (2026) · CBE—Life Sciences Education · doi:10.1187/cbe.24-09-0241 · Cited by 3 · see citing works
Assessing the Load: Effects of Visual Representation and Task Features on Exam Performance in Undergraduate Molecular Life Sciences
Arneson, J. B., & Offerdahl, E. G. (2022) · Research in Science Education · doi:10.1007/s11165-022-10057-7 · Cited by 3 · see citing works
Forming Groups in a Large-Enrollment Biology Class: Group Permanence Matters More than Group Size
Connell, G. L., Donovan, D. A., & Theobald, E. J. (2023) · CBE—Life Sciences Education · doi:10.1187/cbe.22-08-0172 · Cited by 3 · see citing works
Self-citation
A Look at Demographics and Transition to Virtual Assessments: An Analysis of Bias in the American Board of Surgery General Surgery Certifying Exams
Ibáñez, B., Jones, A. T., Jeyarajah, D. R., Dent, D. L., Prendergast, C. O., & Barry, C. L. (2024) · Journal of surgical education · doi:10.1016/j.jsurg.2024.01.001 · Cited by 3 · see citing works
Community college biology students' understanding of Vision and Change core concepts
Courtney, N. A., Smith, M. K., & Esparza, D. (2025) · Journal of Microbiology and Biology Education · doi:10.1128/jmbe.00211-24 · Cited by 5 · see citing works
Open-Inquiry vs. Broadly Relevant Short-Term Research Experiences for Non-Biology Majors
Hebert, S. L., Blum, J. E., Wassenberg, D. M., Marks, D. I., Barry, K., & Cotner, S. H. (2021) · Journal of Microbiology and Biology Education · doi:10.1128/jmbe.v22i1.2167 · Cited by 3 · see citing works
Mind the Gap: Investigating Disaggregated Student Performance in an Upper-Division Biochemistry Course Following Team-Based Learning
Woodbury, J., & Offerdahl, E. G. (2024) · CBE—Life Sciences Education · doi:10.1187/cbe.24-02-0090 · Cited by 2 · see citing works
A school-based program to prevent depressive symptoms and strengthen well-being among pre-vocational students (Happy Lessons): protocol for a cluster randomized controlled trial and implementation study
Rombouts, M., Duinhof, E. L., Kleinjan, M., Kraiss, J. T., Shields‐Zeeman, L., & Monshouwer, K. (2022) · BMC Public Health · doi:10.1186/s12889-021-12321-3 · Cited by 3 · see citing works
Increased Pass Rates in Introductory Biology: Benefits and Potential Costs of Implementing a Mathematics Prerequisite in a Community College Setting
Kiser, S., Andrews, C. M., Seidel, S. B., Fisher, M. R., Wright, N. A., & Theobald, E. J. (2022) · CBE—Life Sciences Education · doi:10.1187/cbe.21-09-0248 · Cited by 2 · see citing works
Self-citation
Influence of an oral health promotion program on the evolution of dental status in New Caledonia: A focus on health inequities
Skandrani, A., Pichot, H., Pegon-Machat, E., Pereira, B., & Tubert‐Jeannin, S. (2023) · PLoS ONE · doi:10.1371/journal.pone.0287067 · Cited by 2 · see citing works
Comparing Academic Performance of Elementary Education Majors in General Education Science Courses
Nixon, R. S., & Bailey, E. G. (2025) · PLoS ONE · doi:10.1371/journal.pone.0320137 · Cited by 1 · see citing works
Grade Disparities Associated with Student Demographics in Upper-Division Biology Courses Across Multiple Institutions
Farrar, V. S., Medinaceli, M. V., Young, N. T., Bonem, E. M., Mead, C., Matz, R. L., & Caporale, N. (2025) · CBE—Life Sciences Education · doi:10.1187/cbe.24-02-0075 · Cited by 1 · see citing works
Nine Undergraduate Biology Instructors Revealing their LGBTQ+ Identities in Class Resulted in Benefits for their LGBTQ+ Students and Students with Other Marginalized Identities
Busch, C. A., Bhanderi, P. B., Cooper, K. M., & Brownell, S. E. (2025) · CBE—Life Sciences Education · doi:10.1187/cbe.24-10-0254 · Cited by 1 · see citing works
Overcoming the more measurements, the better intuitive heuristic in physics data evaluation: The role of inhibitory control
Hong, W., Li, T., Wang, L., Dai, R., Bao, L., Yu, K., & Xiao, Y. (2025) · Physical Review Physics Education Research · doi:10.1103/lw7v-86jq · Cited by 1 · see citing works
A Non-Volunteer CURE in Introductory Biology Does Not Increase Student Retention at a Regional Comprehensive University
Laporte, M., & Casper, A. M. (2026) · CBE—Life Sciences Education · doi:10.1187/cbe.25-09-0213 · Cited by 1 · see citing works
The Impact of Active Learning on Student Course Performance in STEM Varies by Type and Intensity: A Meta-Analysis
Xu, S., Velasco, V., Hill, M. J., Tran, E. T., Agrawal, S., Arroyo, E. N., Behling, S., Chambwe, N., Cintrón, D. L., Cooper, J. D., et al. (2026) · Education Sciences · doi:10.3390/educsci16071044 · Cited by 4 · see citing works
Self-citation
Assessing Community College Biology Student Perceptions of Being Called on in Class
Alvares, S. M., Shlichta, J. G., McFarland, J., & Theobald, E. J. (2023) · CBE - Life Sciences Education · doi:10.1187/cbe.23-05-0068
Self-citation
Investigating the link between students’ learning intentions and their learning outcomes: A clustered randomised controlled trial
Ersoy, E., Forshaw, R., & Lampert, S. (2024) · International Review of Economics Education · doi:10.1016/j.iree.2024.100299 · Cited by 1 · see citing works
National randomized experimental study finds no evidence of undergraduate bias against LGB biology instructors, even in LGB-unfriendly states
Busch, C. A., Bhanderi, P. B., Brownell, S. E., & Cooper, K. M. (2025) · BioScience · doi:10.1093/biosci/biaf040 · Cited by 1 · see citing works
Design Plan for an Evolution Study of Related Characteristics of a Population
Rodríguez‐Díaz, J. M., Pruneda, R. E., & Rodríguez-Hernández, M. (2022) · Mathematics · doi:10.3390/math10050792 · Cited by 1 · see citing works
Active learning’s impact on student course performance in STEM varies by type and intensityPreprint
Xu, S.-M., Velasco, V., Hill, M. J., Tran, E. T., Agrawal, S., Arroyo, E., Behling, S., Chambwe, N., Cintrón, D. L., Cooper, J. D., et al. (2025) · bioRxiv · doi:10.1101/2025.06.01.657285 · Cited by 1 · see citing works
Self-citation

Authors: Theobald, E.
Affiliations: University of Washington (US)
Corresponding author: Theobald, Elli
ORCID: Theobald, Elli
Venue: CBE—Life Sciences Education
Publisher: American Society for Cell Biology (ASCB)
Published: 2018-09
Volume: 17
Issue: 3
Pages: rm2
ISSN: 1931-7913 (ISSN-L)
DOI: 10.1187/cbe.17-12-0280
PMC: PMC6234830
OpenAlex: W2888039885
Cited by: 114
References: 25
Citations by year: 2026: 11 · 2025: 13 · 2024: 15 · 2023: 17 · 2022: 16 · 2021: 23 · 2020: 9 · 2019: 9 · 2018: 1
FWCI: 8.71
Type: journal article
Language: en
Open access: hybrid
doi.org
Licence: cc-by-nc-sa
Locations: CBE—Life Sciences Education (publishedVersion, cc-by-nc-sa)
Europe PMC (PubMed Central) (publishedVersion, cc-by-nc-sa)
PubMed Central (publishedVersion, cc-by-nc-sa)
PubMed (publishedVersion)
Funders: University of Washington
Topics: Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
Physical Sciences › Mathematics › Statistics and Probability › Statistics Education and Methodologies
Social Sciences › Education › Educational Research and Methods
Keywords: random effects, multilevel regression, student clustering, treatment implementation, discipline-based education research
MeSH terms: Humans, Random Allocation, Research Design, Science, Students, Cluster Analysis, Multilevel Analysis

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Students Are Rarely Independent: When, Why, and How to Use Random Effects in Discipline-Based Education Research

Abstract

Integrity signals

Claims the paper makes

Claims other papers make about this one

Self-check

Cites

Cited by

Related

Record

About these checks