A Randomized Assessment of Online Learning
The DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index.
Abstract
A microeconomics principles course employing random assignment across three sections with different teaching models is used to explore learning outcomes as measured by a cumulative final exam for students who participate in traditional face-to-face classroom instruction, blended face-to-face and online instruction with reduced instructor contact time, and a purely online instructional format. Evidence indicates learning outcomes were reduced for students in the purely online section relative to those in the face-to-face format by 5 to 10 points on a cumulative final exam. No statistically significant differences in outcomes are observed for students in the blended relative to the face-to-face section.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The text we found for this paper looks like another document, so the self-check wasn’t run.
Citations
Cites
Can Web Courses Replace the Classroom in Principles of Microeconomics?
Brown, B. W., & Liedholm, C. E. (2002) · American Economic Review · doi:10.1257/000282802320191778 · Cited by 408
Interactive Learning Online at Public Universities: Evidence from a Six‐Campus Randomized Trial
Bowen, W. G., Chingos, M. M., Lack, K. A., & Nygren, T. I. (2014) · Journal of Policy Analysis and Management · doi:10.1002/pam.21728 · Cited by 403
Figlio, D., Rush, M., & Yin, L. (2013) · Journal of Labor Economics · doi:10.1086/669930 · Cited by 367
Joyce, T., Crockett, S., Jaeger, D. A., Altındağ, O., & O'Connell, S. D. (2015) · Economics of Education Review · doi:10.1016/j.econedurev.2015.02.007 · Cited by 149
Drydakis, N. (2015) · Anglia Ruskin Research Online (Anglia Ruskin University) · Cited by 22
Cited by
Virtual Classrooms: How Online College Courses Affect Student Success
Bettinger, E. P., Fox, L., Loeb, S., & Taylor, E. S. (2017) · American Economic Review · doi:10.1257/aer.20151193 · Cited by 441
Education Technology: An Evidence-Based Review
Escueta, M., Quan, V., Nickow, A. J., & Oreopoulos, P. (2017) · doi:10.3386/w23744 · Cited by 389
Compensating for academic loss: Online learning and student performance during the COVID-19 pandemic
Clark, A. E., Nong, H., Zhu, H., & Zhu, R. (2021) · China Economic Review · doi:10.1016/j.chieco.2021.101629 · Cited by 271
“Second, this study adds to the literature on experimental evidence on the effects of instruction models on student performance (Figlio et al., 2013; Bowen et al., 2014; Alpert et al., 2016; Bettinger et al., 2017; Kozakowski, 2019).”
ResultCheating in online courses: Evidence from online proctoring
Dendir, S., & Maxwell, R. S. (2020) · Computers in Human Behavior Reports · doi:10.1016/j.chbr.2020.100033 · Cited by 257
Upgrading Education with Technology: Insights from Experimental Research
Escueta, M., Nickow, A. J., Oreopoulos, P., & Quan, V. (2020) · Journal of Economic Literature · doi:10.1257/jel.20191507 · Cited by 205
Chirikov, I., Semenova, T., Maloshonok, N., Bettinger, E., & Kizilcec, R. F. (2020) · Science Advances · doi:10.1126/sciadv.aay5324 · Cited by 184
Tate, T., & Warschauer, M. (2022) · Educational Psychologist · doi:10.1080/00461520.2022.2062597 · Cited by 167
“Research has shown that undergraduate students generally performed worse in online compared to face-to-face classes when controlling for population differences (Alpert et al., 2016; Jaggars & Xu, 2010; Xu & Jaggars, 2011, 2013, 2014; Xu & Xu, 2019).”
BackgroundCan Online Delivery Increase Access to Education?
Goodman, J., Melkers, J., & Pallais, A. (2019) · Journal of Labor Economics · doi:10.1086/698895 · Cited by 155
“The online format generally leads to worse learning outcomes (Joyce et al. 2015; Alpert, Couch, and Harmon 2016; Krieg and Henson 2016), particularly for academically weaker students, such as those in community colleges (Xu and Jaggars 2014) and for-pro fi t colleges (Bet-tinger et al. 2017).”
Bird, K. A., Castleman, B. L., & Lohner, G. (2022) · AERA Open · doi:10.1177/23328584221081220 · Cited by 140
“Most like our analysis is the work of Altindag et al. (2021), which leverages data from a public university to investigate the performance of students in in-person versus online courses during the pandemic.”
BackgroundThe impacts of remote learning in secondary education during the pandemic in Brazil
Lichand, G., Doria, C. A., Leal-Neto, O., & Fernandes, J. P. C. (2022) · Nature Human Behaviour · doi:10.1038/s41562-022-01350-6 · Cited by 139
Policy Changes in Global Higher Education: What Lessons Do We Learn from the COVID-19 Pandemic?
Tilak, J. B. G., & Kumar, A. G. (2022) · Higher Education Policy · doi:10.1057/s41307-022-00266-0 · Cited by 128
Zooming to Class? Experimental Evidence on College Students’ Online Learning during COVID-19
Kofoed, M. S., Gebhart, L., Gilmore, D., & Moschitto, R. (2024) · American Economic Review: Insights · doi:10.1257/aeri.20230077 · Cited by 125
Spencer, D., & Temple, T. (2021) · Online Learning · doi:10.24059/olj.v25i2.2227 · Cited by 67
Evaluating the flipped classroom: A randomized controlled trial
Wozny, N., Balser, C., & Ives, D. (2018) · The Journal of Economic Education · doi:10.1080/00220485.2018.1438860 · Cited by 68
Altindag, D. T., Filiz, E. S., & Tekin, E. (2025) · Educational Evaluation and Policy Analysis · doi:10.3102/01623737241274802 · Cited by 60
“Similarly, Alpert et al. (2016) found that students in the purely online section of a course received significantly lower grades than those in the FtF sections or the compressed sections with no significant differences between the latter two versions.”
Lim, J. R. N., Rosenthal, S., Sim, Y. J. M., Lim, Z.-Y., & Oh, K. R. (2021) · Technology Pedagogy and Education · doi:10.1080/1475939x.2021.1934102 · Cited by 90
“…arguments about the successful deployment of online courses and content (Harasim, 2017; Shea & Bidjerano, 2009), there is mixed evidence of that success (Alpert et al., 2016; Hart et al., 2019), and getting students to engage with online learning can be a challenge (Panigrahi et al., 2018).”
BackgroundBulman, G. B., & Fairlie, R. W. (2022) · Education Finance and Policy · doi:10.1162/edfp_a_00384 · Cited by 81
Baumeister, R. F., Tice, D. M., & Bushman, B. J. (2023) · Perspectives on Psychological Science · doi:10.1177/17456916221121815 · Cited by 53
“, Alpert et al., 2016; Halloran et al., 2021; Kofoed et al., 2021; Cellini & Grueso, 2021). Overall, online learning has been shown to be less engaging (e.g., Kop et al., 2011) and lead to less learning (e.g., Heppen et al., 2017) than traditional in-person learning. Even in a best-case scenario such as The Netherland…”
ResultMcPartlan, P., Rutherford, T. A., Rodriguez, F., Shaffer, J. F., & Holton, A. J. (2021) · The Internet and Higher Education · doi:10.1016/j.iheduc.2021.100793 · Cited by 65
Learning in the time of Covid-19: Some preliminary findings
Engelhardt, B., Johnson, M., & Meder, M. E. (2021) · International Review of Economics Education · doi:10.1016/j.iree.2021.100215 · Cited by 53
Innovating teachers’ professional learning through digital technologies
Minea-Pic, A. (2020) · OECD education working papers · doi:10.1787/3329fae9-en · Cited by 101
Student Learning in Online College Programs
Cellini, S. R., & Grueso, H. (2021) · AERA Open · doi:10.1177/23328584211008105 · Cited by 54
“To assess student outcomes in online courses, several studies randomly assign students to either an online or inperson format of a single college course and examine performance on final exams or course grades (Alpert et al., 2016; Bowen et al., 2014, Figlio et al., 2013; Joyce et al., 2015).”
BackgroundOnline teaching, procrastination and student achievement
De Paola, M., Gioia, F., & Scoppa, V. (2023) · Economics of Education Review · doi:10.1016/j.econedurev.2023.102378 · Cited by 38
Effects of Flipped Classroom Instruction: Evidence from a Randomized Trial
Setren, E. M., Greenberg, K., Moore, O., & Yankovich, M. (2020) · Education Finance and Policy · doi:10.1162/edfp_a_00314 · Cited by 55
Mead, C., Supriya, K., Zheng, Y., Anbar, A. D., Collins, J. P., LePore, P. C., & Brownell, S. E. (2020) · PLoS ONE · doi:10.1371/journal.pone.0243916 · Cited by 42
“Our finding is consistent with some prior work [11, 12, 33, 34], but not all [14, 32].”
ResultAddressing Non-Financial Barriers to College Access and Success: Evidence and Policy Implications
Dynarski, S., Nurshatayeva, A., Page, L., & Scott-Clayton, J. (2022) · doi:10.3386/w30054 · Cited by 38
Klasik, D., Blagg, K., & Pekor, Z. (2018) · Social Sciences · doi:10.3390/socsci7090165 · Cited by 45
“The students who are the least prepared for college perform worse in online courses than in face-to-face formats (Alpert et al. 2016; Bettinger and Loeb 2017).”
BackgroundWang, Z., Zhang, L., Liu, Y., Jiang, W., Jia, J., Tang, S., & Liu, X. (2021) · Infectious Diseases of Poverty · doi:10.1186/s40249-021-00855-y · Cited by 45
Benefits of additional online practice opportunities in higher education
Schwerter, J., Dimpfl, T., Bleher, J., & Murayama, K. (2021) · The Internet and Higher Education · doi:10.1016/j.iheduc.2021.100834 · Cited by 48
Liao, H., Ma, S., & Xue, H. (2022) · China Economic Review · doi:10.1016/j.chieco.2022.101847 · Cited by 66
Ortagus, J. C., & Yang, L. (2017) · Research in Higher Education · doi:10.1007/s11162-017-9490-y · Cited by 47
Xu, D., & Xu, Y. (2020) · Higher education · doi:10.1007/978-3-030-31365-4_10 · Cited by 49
Why Did Students Report Lower Test Anxiety during the COVID-19 Pandemic?
Ewell, S. N., Josefson, C. C., & Ballen, C. J. (2022) · Journal of Microbiology & Biology Education · doi:10.1128/jmbe.00282-21 · Cited by 23
Hachey, A. C., Conway, K. M., Wladis, C., & Karim, S. (2022) · Journal of Computing in Higher Education · doi:10.1007/s12528-022-09319-0 · Cited by 32
“Similar findings of lower grades and negative impact on later enrollment are reported for both baccalaureate and community college students across a variety of disciplines (Alpert et al., 2016; Burns et al., 2013; Faidley, 2018; Francis et al., 2019; Garman, 2012; Hart et al., 2018; Keramidas, 2012).”
ResultFischer, C., Baker, R., Li, Q., Orona, G. A., & Warschauer, M. (2022) · Educational Evaluation and Policy Analysis · doi:10.3102/01623737211055768 · Cited by 32
“As the few studies that have done so have only assigned students to a single online course (Alpert et al., 2016; Bowen et al., 2014; Figlio et al., 2013; Joyce et al., 2015), it is difficult to estimate causal effects of online course taking on students’ more distal college outcomes.”
BackgroundCan Online Delivery Increase Access to Education?
Goodman, J., Melkers, J. E., & Pallais, A. (2016) · National Bureau of Economic Research · doi:10.3386/w22754 · Cited by 46
Self-Regulated Learning and Academic Success in Online College Learning
Ma, L., & She, L. (2023) · The Asia-Pacific Education Researcher · doi:10.1007/s40299-023-00748-8 · Cited by 41
“Recent studies have reported significantly decreased academic performance and course retention in the context of online courses across countries (Alpert et al., 2016; Joyce et al., 2015; Xu & Jaggars, 2013), and these adverse effects in online education are more pronounced among college learners…”
Kozakowski, W. (2019) · Economics of Education Review · doi:10.1016/j.econedurev.2019.03.004 · Cited by 34
Orona, G. A., Li, Q., McPartlan, P., Bartek, C., & Xu, D. (2022) · Computers & Education · doi:10.1016/j.compedu.2022.104498 · Cited by 27
Yang, N., & Ghislandi, P. (2024) · Higher Education · doi:10.1007/s10734-023-01173-y · Cited by 32
“Specifically, for course grades, students in the online courses had significantly lower rates (Hart et al., 2018; Roughton, 2018; Alpert et al., 2016; Figlio et al., 2013; Lee et al., 2022; Huntington-Klein et al., 2017; Johnson & Mejia, 2014; Kaupp, 2012; Sublett, 2019; Xu & Jaggars, 2013)…”
Distance Learning in Higher Education: Evidence from a Randomized Experiment
Cacault, M. P., Hildebrand, C., Laurent-Lucchetti, J., & Pellizzari, M. (2021) · Journal of the European Economic Association · doi:10.1093/jeea/jvaa060 · Cited by 84
“For example, McPherson and Bacow (2015) and Banerjee and Duflo (2014) document high drop-out rates for MOOCs and Bettinger et al. (2017) shows worse labor market outcomes for graduates of online for-profit colleges. Brown and Liedholm (2002) show suggestive evidence of substantially lower achievement for college stude…”
BackgroundInfluentialAddressing nonfinancial barriers to college access and success: Evidence and policy implications
(2023) · Handbook of the Economics of Education · doi:10.1016/bs.hesedu.2022.11.007 · Cited by 21
Online or on-campus? Analysing the effects of financial education on student knowledge gain
Agasisti, T., Barucci, E., Cannistrà, M., Marazzina, D., & Soncin, M. (2023) · Evaluation and Program Planning · doi:10.1016/j.evalprogplan.2023.102273 · Cited by 33
Chingos, M. M., Griffiths, R. J., & Mulhern, C. (2017) · Journal of Research on Educational Effectiveness · doi:10.1080/19345747.2017.1300362 · Cited by 21
“In addition, more recent experiments have failed to detect significant improvements (or decrements) associated with adoption of technology in hybrid formats, which combine face-to-face and online instruction (Alpert et al., 2016; Bowen, Chingos, Lack, & Nygren, 2014; Chingos, Griffiths, Mulhern, & Spies, 2017).”
BackgroundMartin, A. J. (2023) · Australian Journal of Education · doi:10.1177/00049441231179791 · Cited by 28
“This runs counter to some prior research which has found remote learning associated with poorer academic outcomes (Alpert et al., 2016), elevated academic distress (Horita et al., 2021), decreased motivation (Marzoli et al., 2021), and learning loss (Shin & Hickey, 2021).”
BackgroundResultHarmon, O. R., & Tomolonis, P. (2019) · International Review of Economics Education · doi:10.1016/j.iree.2019.100157 · Cited by 19
Self-citationBulman, G. B., & Fairlie, R. W. (2021) · National Bureau of Economic Research · doi:10.3386/w28715 · Cited by 29
In the Storm of COVID-19: College Students’ Perceived Challenges With Virtual Learning
Davis, L., Sun, Q., Lone, T., Levi, A., & Xu, P. (2022) · Journal of Higher Education Theory and Practice · doi:10.33423/jhetp.v22i1.4964 · Cited by 22
E-learning with multiple-try-feedback: Can hints foster students’ achievement during the semester?
Schwerter, J., Wortha, F., & Gerjets, P. H. (2022) · Educational Technology Research and Development · doi:10.1007/s11423-022-10105-z · Cited by 23
“…in higher education has focused on digital (or virtual) learning formats, face-to-face formats, and blended learning formats, a hybrid of the two (Alpert et al., 2016; Bettinger et al., 2017; Bowen et al., 2014; Brown & Liedholm, 2002; Coates et al., 2004; Figlio et al., 2013; Jaggars & Xu,…”
Blaskovits, F., Bayoumi, I., Davison, C., Watson, A., & Purkey, E. (2023) · BMC Public Health · doi:10.1186/s12889-023-15010-5 · Cited by 28
“These have been beneficial in other settings [35, 36].”
BackgroundProviding MOOCs: A FUN way to enroll students?
Jacqmin, J. (2018) · Information Economics and Policy · doi:10.1016/j.infoecopol.2018.10.002 · Cited by 22
The effect of educational technology on college students’ labor market performance
Lu, Y., & Song, H. (2020) · Journal of Population Economics · doi:10.1007/s00148-019-00756-3 · Cited by 18
“In particular, several studies focus on online courses—a specific type of ICT adoption—and examine their impact on students’ academic performance; most find a zero or negative effect (Fairlie and Robinson 2013; Figlio et al. 2013; Alpert et al. 2016; Bettinger et al. 2017).”
BackgroundHadar, L. L., Alpert, B., & Ariav, T. (2020) · Prospects · doi:10.1007/s11125-020-09516-8 · Cited by 22
Geospatial Analysis in Higher Education Research
Hillman, N. (2017) · Higher education · doi:10.1007/978-3-319-48983-4_11 · Cited by 21
Perspectives on facilitating dynamic ecology courses online using active learning
Gahl, M. K., Gale, A., Kaestner, A., Yoshina, A., Paglione, E., & Bergman, G. (2020) · Ecology and Evolution · doi:10.1002/ece3.6953 · Cited by 17
“…for students in active learning classes (Freeman et al., 2014; Styers et al., 2018), several studies suggest that students receive lower grades and have lower completion rates in the online version of a course compared to the face-to-face version (Alpert et al., 2016; Bettinger et al., 2017).”
BackgroundKoth, A. J., Focken, A. G., Lyden, E. R., & Yoachim, S. D. (2021) · Journal of Dental Education · doi:10.1002/jdd.12757 · Cited by 21
Cheung, Y. Y. H., Lam, K. F., Zhang, H., Kwan, C. W., Wat, K. P., Zhang, Z., Zhu, K., Chung, Y., & Yin, G. (2023) · Frontiers in Education · doi:10.3389/feduc.2023.1160430 · Cited by 22
“Small to medium-sized randomized controlled experiments were also conducted to compare face-to-face, blended, and asynchronous online teaching (Alpert et al., 2016; Reavley et al., 2018; Crawford et al., 2021; Jiang et al., 2021).”
BackgroundOjha, M., & Rahman, M. A. (2021) · Education Policy Analysis Archives · doi:10.14507/epaa.29.5919 · Cited by 22
Salient syllabi: Examining design characteristics of science online courses in higher education
Fischer, C., McPartlan, P., Orona, G. A., Yu, R., Xu, D., & Warschauer, M. (2022) · PLoS ONE · doi:10.1371/journal.pone.0276839 · Cited by 21
The effects of online vs in-class testing in moderate-stakes college environments
Hill, A., & LoPalo, M. (2023) · Economics of Education Review · doi:10.1016/j.econedurev.2023.102505 · Cited by 19
The Role and Influence of Exclusively Online Degree Programs in Higher Education
Ortagus, J. C., Hughes, R., & Allchin, H. (2024) · American Educational Research Journal · doi:10.3102/00028312231222264 · Cited by 20
“Students in face-to-face instruction have earned higher course grades than their peers in online courses in several experimental studies (Alpert et al., 2016; Figlio et al., 2013; Joyce et al., 2015).”
BackgroundPaola, M. D., Gioia, F., & Scoppa, V. (2022) · Social Science Research Network · doi:10.2139/ssrn.4114561 · Cited by 15
“Some papers rely on small-scale experiments where students are randomly assigned to alternative teaching systems (face-to-face, online and blended) (Alpert et al. 2016, Bowen et al. 2014, Coates et al. 2004, Cacault et al., 2021; Figlio et al. 2013, Joyce et al. 2015) and show negative effects of…”
BackgroundReturns to Effort: Experimental Evidence from an Online Language Platform
Ersoy, F. Y. (2019) · Experimental Economics · doi:10.1007/s10683-020-09689-1 · Cited by 15
Live or Lecture Capture: Evidence from a Classroom Random Control Trial
Artz, B., Johnson, M., Robson, D., & Taengnoi, S. (2021) · International Review of Economics Education · doi:10.1016/j.iree.2022.100240 · Cited by 13
Driessen, E. P., Beatty, A. E., & Ballen, C. J. (2022) · PLoS ONE · doi:10.1371/journal.pone.0273185 · Cited by 16
Camacho, I., & Goel, A. K. (2018) · ACM Conference on Learning @ Scale (L@S) · doi:10.1145/3231644.3231679 · Cited by 11
“Previous research has found that online programs have generally led to worse learning outcomes [1].”
BackgroundBehavioral Engagement Shifts Among At-Risk High School Students Enrolled in Online Courses
Darling‐Aduana, J. (2019) · AERA Open · doi:10.1177/2332858419887736 · Cited by 23
Xu, D., & Xu, Y. (2020) · Higher education · doi:10.1007/978-3-030-11743-6_10-1 · Cited by 17
Assessing the effect of Massive Open Online Courses as remedial courses in higher education
Agasisti, T., AZZONE, G. F., & Soncin, M. (2021) · Innovations in Education and Teaching International · doi:10.1080/14703297.2021.1886969 · Cited by 12
“Alpert et al. (2016) compared learning outcomes for traditional, blended and purely online formats in the USA, finding that online courses compared negatively with traditional education, most especially for disadvantaged students.”
BackgroundCrisis-driven digitalization and academic success across disciplines
Tinjić, D., & Nordén, A. (2024) · PLoS ONE · doi:10.1371/journal.pone.0293588 · Cited by 20
“Conversely, several studies favor conventional face-to-face learning [23, 30, 32, 39–42], suggesting that traditional learning models yield somewhat superior student outcomes [30, 32, 35, 40], but no significant changes in attendance, dropout rates, or the time spent on assigned tasks [32].”
The Costs, Quality, and Scalability of Blended Learning in Postgraduate Management Education
Soncin, M., Agasisti, T., Frattini, F., Patrucco, P. P., & Pero, M. E. P. (2022) · Organizational Behavior Teaching Review · doi:10.1177/10525629221103826 · Cited by 17
“Most of the debate has been focused on its effectiveness in this setting, without considering the cost of delivery (Alpert et al., 2016; Bettinger et al., 2015; Figlio et al., 2013; Joyce et al., 2015).”
BackgroundCOVID-19 and Education: A Survey of the Research
Hinrichs, P. L. (2021) · Economic Commentary (Federal Reserve Bank of Cleveland) · doi:10.26509/frbc-ec-202104 · Cited by 14
Fifteen Common but Questionable Principles of Multimedia Learning
Feldon, D. F., Jeong, S., & Clark, R. E. (2021) · Cambridge University Press eBooks · doi:10.1017/9781108894333.005 · Cited by 15
“…experiments have been conducted on this issue, the findings from such studies consistently suggest that online courses yield negative or null advantages for learning outcomes compared to face-to-face courses (e.g., Alpert, Couch, & Harmon, 2016; Bowen et al., 2014; Figlio, Rush, & Yin, 2013).”
BackgroundBohman, A., Eger, M. A., Hjerm, M., & Mitchell, J. (2023) · European Journal of Higher Education · doi:10.1080/21568235.2023.2209707 · Cited by 17
“…the di ffi culties associated with student engagement in online learning (Dixson 2010), we expect a university-wide shift to that mode of instruction to have negative consequences for students (Alpert, Couch, and Harmon 2016; Bettinger et al. 2017), and especially for students in their fi rst year.”
Perception of teaching performance in the virtual learning environment
Morales-Romero, G., Trinidad-Loli, N., Caycho-Salas, B., Manrique, Y. R. P., León-Velarde, C., Gamarra-Mendoza, S., Alvarado-Bravo, N., Torres-Quiroz, A., Aliaga-Valdez, C., & Aldana-Trejo, F. (2021) · International Journal of Evaluation and Research in Education (IJERE) · doi:10.11591/ijere.v10i4.22056 · Cited by 9
“In order for them to be reached by students, it is necessary for teachers to have a profile that allows them to achieve optimal performance, from the perspective of university students [24], [25].”
BackgroundLearning Analytics in Education for the Twenty-First Century
Witte, K. D., & Chénier, M.-A. (2022) · Springer eBooks · doi:10.1007/978-3-031-16624-2_16 · Cited by 17
How Do Online Degrees Affect Labor Market Prospects? Evidence from a Correspondence Audit Study
Lennon, C. (2020) · Industrial and Labor Relations Review · doi:10.1177/0019793919899943 · Cited by 14
“A notable exception in the literature is Alpert, Couch, and Harmon (2016), who found that performance on a cumulative microeconomics final exam was 5 to 10 points (out of 100) lower for students randomly assigned to an online section.”
BackgroundThe access‐equity dualism of online learning in community colleges
Sublett, C. (2022) · New Directions for Community Colleges · doi:10.1002/cc.20508 · Cited by 10
Ewell, S. N., Cotner, S. H., Drake, A. G., Fagbodun, S., Google, A. N., Robinson, L., Soneral, P. A. G., & Ballen, C. J. (2022) · Journal of Microbiology and Biology Education · doi:10.1128/jmbe.00260-21 · Cited by 21
Petersen, L., & Rholes, R. (2022) · Journal of Economic Dynamics and Control · doi:10.1016/j.jedc.2022.104460 · Cited by 9
“, 2020 ) and that learning outcomes are worse online ( Alpert et al., 2016; Bettinger et al., 2017; Cacault et al., 2021 ). Our experiment is presented in a highly contextualized manner. We present public signals to subjects as forecasts from a central bank and fully describe to them the economic system, including the…”
InfluentialGuo, J., King, R. B., Ding, Q., & Fan, M. (2022) · Education Sciences · doi:10.3390/educsci12070465 · Cited by 16
“Randomized control experiments were conducted to make causal inference about the unbiased effect of those interventions across different learning environments [54,55].”
MethodComparing the Effects of SPOC and Face-to-Face Courses and Factors Influencing the Effects
Han, J., & Shi, J. (2022) · International Journal of Information and Education Technology · doi:10.18178/ijiet.2022.12.6.1654 · Cited by 8
“categories on the effectiveness of online learning [4], [20].”
BackgroundBusiness school grades, assessment scores, and course withdrawals in the Covid-19 pandemic
Engelhardt, B., Johnson, M., & Taengnoi, S. (2022) · Journal of Education for Business · doi:10.1080/08832323.2022.2109563 · Cited by 14
“Alpert, Couch, and Harmon (2016) attempt to disentangle this problem with a randomized control trial assessing learning outcomes in face-to-face, blended, and online courses.”
BackgroundIn‐person versus online instruction: Evidence from principles of economics
Elzinga, K. G., & Harper, D. Q. (2023) · Southern Economic Journal · doi:10.1002/soej.12635 · Cited by 11
Testing the effects of adaptive learning courseware on student performance: An experimental approach
Eau, G., Hoodin, D., & Musaddiq, T. (2021) · Southern Economic Journal · doi:10.1002/soej.12547 · Cited by 7
Experiences with online education during COVID-19
Norris, C., & Fenwick, J. B. (2022) · ACM Southeast Regional Conference (ACMSE) · doi:10.1145/3476883.3524049 · Cited by 8
“In another study [2], students were randomly assigned to either a purely online course, a purely face-to-face course, or a blend of online and face-to-face.”
BackgroundResultHolistic Online Learning, in a Post COVID-19 World
Jackson, K. M., & Szombathelyi, M. K. (2022) · Acta Polytechnica Hungarica · doi:10.12700/aph.19.11.2022.11.7 · Cited by 11
Low-Cost Randomized Controlled Trials in Education
Wozny, N., Balser, C., & Ives, D. (2018) · AEA Papers and Proceedings · doi:10.1257/pandp.20181054 · Cited by 7
Fostering higher-order thinking skills online in higher education: A scoping review
Hague, C. (2024) · OECD education working papers · doi:10.1787/84f7756a-en · Cited by 32
Transitioning to Online Learning amid COVID-19: Perspectives in a Civil Engineering Program
Maraqa, M. A., Hamouda, M. A., El-Hassan, H., El-Dieb, A., & Hassan, A. A. (2022) · Online Learning · doi:10.24059/olj.v26i3.2616 · Cited by 11
“, 2009; Reynolds & Paulus, 2009), remained the same (Alpert et al., 2016; Bowen et al., 2014), or decreased (Wellington et al.”
BackgroundResultInfluentialZeng, C. (2023) · International Journal for the Scholarship of Teaching and Learning · doi:10.20429/ijsotl.2023.17107 · Cited by 9
“Supporting this view, Brown and Liedholm (2002) and Alpert et al. (2016) examined face-to-face, online and hybrid learning models and found that students who learn purely online perform worst relative to students who learn via other formats.”
Davidson, G. R., Xiao, H., & Davidson, K. (2024) · Education Sciences · doi:10.3390/educsci14111268 · Cited by 5
Nygren, Å., Alégroth, E., Eriksson, A., & Pettersson, E. (2023) · Education Sciences · doi:10.3390/educsci13020181 · Cited by 7
“While much could be said about the positive aspects of online learning, a sizeable body of research indicate negative academic outcomes among students in online courses as opposed to campus courses [26–29].”
FACE-TO-FACE VS. ONLINE LEARNING in SCIENCE and ENGINEERING COURSES DURING COVID-19 EPOCH
Diez-Pascual, A. M., García-Díaz, P., Capilla, R. P., & Jurado‐Sánchez, B. (2023) · Research in Science & Technological Education · doi:10.1080/02635143.2023.2275136 · Cited by 7
“A few studies claimed that online instruction yields worse student performance than in site coursework (Alpert, Couch, and Harmon 2016).”
School closures and student achievement, evidence from a high stakes exam
Riudavets-Barcons, M., & Uusitalo, R. (2024) · Journal of the Finnish Economic Association · doi:10.33358/jfea.129470 · Cited by 13
On Measuring and Reducing Selection Bias With a Quasi‐Doubly Randomized Preference Trial
Joyce, T., Remler, D. K., Jaeger, D. A., Altındağ, O., O'Connell, S. D., & Crockett, S. (2017) · Journal of Policy Analysis and Management · doi:10.1002/pam.21976 · Cited by 6
The Impact of the COVID-19 Pandemic on Teaching Outcomes in Higher EducationPreprint
Hansen, P., Struth, L., Thon, M., & Umbach, T. (2021) · SSRN Electronic Journal · doi:10.2139/ssrn.3916349 · Cited by 6
“Experimental evidence is provided by Figlio et al. (2013), Joyce et al. (2015), Bowen et al. (2014) and Alpert et al. (2016).”
The Use of Innovative Technologies in Education:
Коренева, І., Myroshnychenko, N., Mykhailenko, L., Matiash, O., & Кузьменко, Г. О. (2023) · Cadernos de Educação Tecnologia e Sociedade · doi:10.14571/brajets.v16.n3.625-638 · Cited by 4
The novel liquid learning system and the online gap in academic performance
Alegría, R., Cárabe, P., Chahoud, A., & Román, A. G. D. S. (2022) · Education Economics · doi:10.1080/09645292.2022.2113860 · Cited by 5
Student performance in online health courses
Speer, J. D. (2023) · Education Economics · doi:10.1080/09645292.2023.2185570 · Cited by 4
Metadata
- Authors
- Alpert, W. T., Couch, K. A., & Harmon, O. R.
- Affiliations
- University of Connecticut (US)
- Venue
- American Economic Review
- Publisher
- American Economic Association
- Published
- 2016-05-01
- Volume
- 106
- Issue
- 5
- Pages
- 378-382
- ISSN
- 0002-8282 (ISSN-L)
- OpenAlex
- W2355830311
- Cited by
- 209
- References
- 5
- Citations by year
- 2026: 10 · 2025: 19 · 2024: 21 · 2023: 33 · 2022: 44 · 2021: 38 · 2020: 15 · 2019: 9 · 2018: 10 · 2017: 8 · 2016: 2
- FWCI
- 42.90
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- American Economic Review (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- Topics
- Social Sciences › Education › Online and Blended Learning
- Social Sciences › Education › Education during COVID-19 pandemic
- Social Sciences › Education › Technology-Enhanced Education Studies
- Keywords
- online learning, blended learning, face-to-face teaching, learning outcomes, final exam performance, course format comparison
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.