The DOI matches its Crossref record; 67 of the 67 references it cites were found in a registry or an index.
Facilitating flexible learning by replacing classroom time with an online learning environment: A systematic review of blended learning in higher education
Open accessAbstract
Higher education institutions are trying to provide more flexibility and individualization, which is mainly realized through the use of new technologies and implemented in online or blended learning designs. This systematic review aims to investigate the impact of replacing classroom time with an online learning environment. The meta-analysis (k = 21 effect sizes) applied strict inclusion criteria concerning research design, measurement of learning outcomes and implementation of blended learning. The estimated effect size (Hedge's g) was positive, although not significantly different from zero and the confidence interval [-0.13, 0.25], suggesting that overall differences between blended and conventional classroom learning are small, and, at best, very small negative or moderate positive effects are plausible. This means that despite a reduction in classroom time between 30 and 79 per cent, equivalent learning outcomes were found. Consequently, blended learning with reduced classroom time is not systematically more or less effective than conventional classroom learning.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: the PRISMA statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & for the PRISMA Group (2009) · BMJ · doi:10.1136/bmj.b2535 · Cited by 83,823
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & The PRISMA Group (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000097 · Cited by 64,042
Preferred reporting items for systematic reviews and meta-analyses: the PRISMA Statement.
(2009) · PubMed Central · Cited by 44,859
(2000) · Cited by 9,198
Blended learning: Uncovering its transformative potential in higher education
Garrison, D. R., & Kanuka, H. (2004) · The Internet and Higher Education · doi:10.1016/j.iheduc.2004.02.001 · Cited by 4,849 · cited 2×
“…publications are those by Graham (2006): “ blended learning systems combine face-to-face instruction with computer-mediated instruction ” (p. 5) and by Garrison and Kanuka (2004): “ the thoughtful integration of classroom face-to-face learning experiences with online learning experiences.”
BackgroundThe Handbook of Research Synthesis and Meta-Analysis
(2019) · doi:10.7758/9781610448864 · Cited by 4,553 · cited 2×
“While no asymmetry was visually apparent, more formal tests for asymmetry (Chapter 3.3.2 in Sutton, 2009) were conducted as well, but these were not significant (linear regression test: p = 0.69, rank correlation test: 0.86).”
MethodInfluentialThe Handbook of Blended Learning: Global Perspectives, Local Designs
Bonk, C. J., Graham, C. R., Cross, J., & Moore, M. G. (2005) · Cited by 2,887
Blended learning systems: Definition, current trends, and future directions
Graham, C. R. (2006) · Cited by 1,765
The effectiveness of online and blended learning: A meta-analysis of the empirical literature
Means, B., Toyama, Y., Murphy, R., & Baki, M. (2013) · Teachers College Record: The Voice of Scholarship in Education · doi:10.1177/016146811311500307 · Cited by 1,638 · cited 9×
“Various factors influence the effectiveness of blended learning, and these can be divided into the moderator categories of methods, practice and condition (Means et al., 2013).”
BackgroundResultInfluentialBlended Learning in Higher Education: Framework, Principles, and Guidelines
Garrison, D. R., & Vaughan, N. D. (2007) · Wiley eBooks · doi:10.1002/9781118269558 · Cited by 1,609
Blended Learning in Higher Education: Framework, Principles, and Guidelines.
Wainwright, S. (2011) · Journal of Physical Therapy Education · doi:10.1097/00001416-201110000-00013 · Cited by 1,438
Blended learning: the new normal and emerging technologies
Dziuban, C., Graham, C. R., Moskal, P. D., Norberg, A., & Sicilia, N. (2018) · International Journal of Educational Technology in Higher Education · doi:10.1186/s41239-017-0087-5 · Cited by 1,252 · cited 2×
“Since practically all universities today use an online learning management system through which at least teaching materials are made available, blended learning is also referred to as “ the new traditional model ” or the “ new normal ” (Dziuban, Graham, Moskal, Norberg, & Sicilia, 2018).”
BackgroundSchwarzer, G., Carpenter, J. R., & Rücker, G. (2015) · Use R! · doi:10.1007/978-3-319-21416-0 · Cited by 1,230
What Do We Mean by Blended Learning?
Hrastinski, S. (2019) · TechTrends · doi:10.1007/s11528-019-00375-5 · Cited by 874
Bernard, R. M., Borokhovski, E., Schmid, R. F., Tamim, R. M., & Abrami, P. C. (2014) · Journal of Computing in Higher Education · doi:10.1007/s12528-013-9077-3 · Cited by 870 · cited 9×
“In the meta-analysis by Bernard et al. (2014), the authors integrated 117 effects of 96 studies between 1990 and 2010 comparing blended learning with classroom instruction in higher education.”
BackgroundMethodInfluentialHandbook of Distance Education
(2018) · doi:10.4324/9781315296135 · Cited by 795
It's not about seat time: Blending, flipping, and efficiency in active learning classrooms
Baepler, P., Walker, J. D., & Driessen, M. (2014) · Computers & Education · doi:10.1016/j.compedu.2014.06.006 · Cited by 778
Shea, P., & Bidjerano, T. (2010) · Computers & Education · doi:10.1016/j.compedu.2010.07.017 · Cited by 758
Nortvig, A.-M., Petersen, A. K., & Balle, S. H. (2018) · The Electronic Journal of e-Learning · Cited by 758
The Challenges of Online Learning: Supporting and Engaging the Isolated Learner
Gillett-Swan, J. (2017) · Journal of Learning Design · doi:10.5204/jld.v9i3.293 · Cited by 747
(2009) · Cited by 670
The applicability of Visible Learning to higher education
Hattie, J. (2015) · Scholarship of Teaching and Learning in Psychology · doi:10.1037/stl0000021 · Cited by 656 · cited 2×
“Regarding the analysis of various forms of university course delivery, Hattie (2015) points out that of greater importance are how teachers – irrespective of the method of delivery – make their success criteria clear and offer challenge and feedback, coupled with the quality of the interaction…”
BackgroundStrelan, P., Osborn, A., & Palmer, E. (2020) · Educational Research Review · doi:10.1016/j.edurev.2020.100314 · Cited by 526
Lim, D. H., & Morris, M. L. (2009) · Educational Technology & Society · Cited by 524
Thai, N. T. T., De Wever, B., & Valcke, M. (2017) · Computers & Education · doi:10.1016/j.compedu.2017.01.003 · Cited by 523
Blending In The Extent and Promise of Blended Education in the United States
Allen, I. E., Seaman, J., & Garrett, R. S. (2007) · Cited by 515
Chen, J., Wang, M., Kirschner, P. A., & Tsai, C.-C. (2018) · Review of Educational Research · doi:10.3102/0034654318791584 · Cited by 450
Going Digital: Shaping Policies, Improving Lives
OECD (2019) · OECD eBooks · doi:10.1787/9789264312012-en · Cited by 435
Interactive Learning Online at Public Universities: Evidence from a Six‐Campus Randomized Trial
Bowen, W. G., Chingos, M. M., Lack, K. A., & Nygren, T. I. (2014) · Journal of Policy Analysis and Management · doi:10.1002/pam.21728 · Cited by 403
Searching for studies: a guide to information retrieval for Campbell systematic reviews
Kugley, S., Wade, A., Thomas, J., Mahood, Q., Jørgensen, A. K., Hammerstrøm, K., & Sathe, N. (2017) · Campbell Systematic Reviews · doi:10.4073/cmg.2016.1 · Cited by 371
Vo, H. M., Zhu, C., & Diep, N. A. (2017) · Studies in Educational Evaluation · doi:10.1016/j.stueduc.2017.01.002 · Cited by 360
Reimagining the new pedagogical possibilities for universities post-Covid-19
Peters, M. A., Rizvi, F., McCulloch, G., Gibbs, P., Görur, R., Hong, M. S., Hwang, Y.-J., Zipin, L., Brennan, M., Robertson, S. L., et al. (2020) · Educational Philosophy and Theory · doi:10.1080/00131857.2020.1777655 · Cited by 325
Defining the nature of blended learning through its depiction in current research
Smith, K., & Hill, J. (2019) · Higher Education Research & Development · doi:10.1080/07294360.2018.1517732 · Cited by 321
The effects of technology use in postsecondary education: A meta-analysis of classroom applications
Schmid, R. F., Bernard, R. M., Borokhovski, E., Tamim, R. M., Abrami, P. C., Surkes, M. A., Wade, C. A., & Woods, J. (2014) · Computers & Education · doi:10.1016/j.compedu.2013.11.002 · Cited by 313
COMPARING STUDENT PERFORMANCE: ONLINE VERSUS BLENDED VERSUS FACE-TO-FACE
Larson, D. K., & Sung, C.-H. (2019) · Online Learning · doi:10.24059/olj.v13i1.1675 · Cited by 234
The promised land of blended learning: Quizzes as a moderator
Spanjers, I. A. E., Könings, K. D., Leppink, J., Verstegen, D. M. L., de Jong, N., Czabanowska, K., & van Merriënboer, J. J. G. (2015) · Educational Research Review · doi:10.1016/j.edurev.2015.05.001 · Cited by 208
Blended, Flipped, and Hybrid Learning: Definitions, Developments, and Directions
Saichaie, K. (2020) · New Directions for Teaching and Learning · doi:10.1002/tl.20428 · Cited by 160
Achievement and Satisfaction in Blended Learning versus Traditional General Health Course Designs
Melton, B. F., Bland, H. W., & Chopak‐Foss, J. (2009) · International Journal for the Scholarship of Teaching and Learning · doi:10.20429/ijsotl.2009.030126 · Cited by 156
Gundlach, E., Richards, K. A. R., Nelson, D. B., & Levesque‐Bristol, C. (2015) · Journal of Statistics Education · doi:10.1080/10691898.2015.11889723 · Cited by 142
Owston, R., & York, D. N. (2017) · The Internet and Higher Education · doi:10.1016/j.iheduc.2017.09.001 · Cited by 141
(2011) · Journal of interactive online learning · Cited by 138
So, H., & Bonk, C. J. (2010) · Cited by 135
Joyce, T., Crockett, S., Jaeger, D. A., Altındağ, O., & O'Connell, S. D. (2015) · Economics of Education Review · doi:10.1016/j.econedurev.2015.02.007 · Cited by 124
Current Research in Blended Learning
Graham, C. R. (2018) · Routledge eBooks · doi:10.4324/9781315296135-15 · Cited by 118
Gagnon, M., Gagnon, J., Desmartis, M., & Njoya, M. M. (2013) · Nursing Education Perspectives · doi:10.5480/10-459 · Cited by 111
Borokhovski, E., Bernard, R. M., Tamim, R. M., Schmid, R. F., & Sokolovskaya, A. (2015) · Computers & Education · doi:10.1016/j.compedu.2015.11.004 · Cited by 102
Time well spent: Creating a community of inquiry in blended first-year writing courses
Hilliard, L., & Stewart, M. K. (2018) · The Internet and Higher Education · doi:10.1016/j.iheduc.2018.11.002 · Cited by 102
The impact of blended and traditional instruction in students’ performance
Vernadakıs, N., Gıannousı, M., Derrı, V., Michalopoulos, M., & Kioumourtzoglou, E. (2012) · Procedia Technology · doi:10.1016/j.protcy.2012.02.098 · Cited by 101
Effect of blended learning environment on transfer of learning: an experimental study
Demirer, V., & Sahin, I. (2013) · Journal of Computer Assisted Learning · doi:10.1111/jcal.12009 · Cited by 96
Adams, A. E. M., Randall, S., & Traustadóttir, T. (2015) · CBE—Life Sciences Education · doi:10.1187/cbe.14-08-0118 · Cited by 67
Flexible Learning in Higher Education
Arfield, J., Hodgkinson, K., Smith, A., & Wade, W. (2013) · Routledge eBooks · doi:10.4324/9781315041575 · Cited by 65
Mahmud, M. M. (2018) · The Journal of AsiaTEFL · doi:10.18823/asiatefl.2018.15.2.7.365 · Cited by 65
Milic, N. M., Trajković, G., Bukumirić, Z., Cirkovic, A. M., Nikolić, I., Milin-Lazovic, J. S., Milic, N., Savic, M. D., Corac, A. M., Marinković, J., et al. (2016) · PLoS ONE · doi:10.1371/journal.pone.0148882 · Cited by 64
The effect of blended courses on student learning: evidence from introductory economics courses
Olitsky, N. H., & Cosgrove, S. B. (2014) · International Review of Economics Education · doi:10.1016/j.iree.2013.10.009 · Cited by 62
Developing a framework for sustainable growth of flexible learning opportunities
Andrade, M. S., & Alden-Rivers, B. (2019) · Higher Education Pedagogies · doi:10.1080/23752696.2018.1564879 · Cited by 60
Chingos, M. M., Griffiths, R. J., Mulhern, C., & Spies, R. R. (2016) · The Journal of Higher Education · doi:10.1080/00221546.2016.1244409 · Cited by 45
Uzun, A., & Şentürk, A. (2010) · Contemporary Educational Technology · doi:10.30935/cedtech/6027 · Cited by 42
The choice of reduced seat time in a blended course
Asarta, C. J., & Schmidt, J. R. (2015) · The Internet and Higher Education · doi:10.1016/j.iheduc.2015.04.006 · Cited by 29
Harrison, D. J., Saito, L., Markee, N. L., & Herzog, S. (2016) · European Journal of Engineering Education · doi:10.1080/03043797.2016.1218826 · Cited by 25
Kennedy, E., Laurillard, D., Horan, B., & Charlton, P. (2015) · Distance Education · doi:10.1080/01587919.2015.1055920 · Cited by 23
Comparison Of Novice Programmers’ Performances: Blended Versus Face-To-Face
Çakıroğlu, Ü. (2012) · Turkish Online Journal of Distance Education · Cited by 17
By design: negotiating flexible learning in the built environment discipline
Tucker, R., & Morris, G. (2012) · Research in Learning Technology · doi:10.3402/rlt.v20i0.14404 · Cited by 15
Effective instruction in APA style in blended and face-to-face classrooms.
Zafonte, M., & Parks‐Stamm, E. J. (2016) · Scholarship of Teaching and Learning in Psychology · doi:10.1037/stl0000064 · Cited by 13
(2014) · Cited by 11
Student Performance in a Principle of Microeconomics Course under Hybrid and Face-to-Face Delivery
Verhoeven, P., & Rudchenko, Т. А. (2013) · American Journal of Educational Research · doi:10.12691/education-1-10-1 · Cited by 11
By design: negotiating flexible learning in the built environment discipline
Tucker, R., & Morris, G. (2012) · Research in Learning Technology · doi:10.3402/rlt.v20i0/14404 · Cited by 10
Cılasun, S. M., Demir-Şeker, S., Dinçer, N. N., & Tekin‐Koru, A. (2018) · Adult Education Quarterly · doi:10.1177/0741713618783890 · Cited by 8
Cited by
Banihashem, S. K., Noroozi, O., den Brok, P., Biemans, H. J. A., & Kerman, N. T. (2023) · The International Journal of Management Education · doi:10.1016/j.ijme.2023.100803 · Cited by 102
Role of AI in Blended Learning: A Systematic Literature Review
Park, Y., & Doo, M. Y. (2024) · The International Review of Research in Open and Distributed Learning · doi:10.19173/irrodl.v25i1.7566 · Cited by 81
Alqarni (2024) · doi:10.1016/j.caeai.2024.100205 · Cited by 62
Exploring the Effectiveness of Flipped Classroom on STEM Student Achievement: A Meta-analysis
Gong, J., Cai, S., & Cheng, M. (2024) · Technology, Knowledge and Learning · doi:10.1007/s10758-023-09700-7 · Cited by 42
Mumcu, B. B., & Çebi, A. (2025) · International Journal of Educational Technology in Higher Education · doi:10.1186/s41239-025-00537-x · Cited by 26
“In higher education, students are afforded flexible and personalized learning opportunities, which are predominantly facilitated through the implementation of online or blended learning environments (Müller & Mildenberger, 2021).”
Metadata
- Authors
- Müller, C., & Mildenberger, T.
- Venue
- Educational Research Review
- Publisher
- Elsevier BV
- Published
- 2021-11
- Volume
- 34
- Pages
- 100394
- ISSN
- 1747-938X (ISSN-L)
- OpenAlex
- W3171442476
- Cited by
- 525
- References
- 67
- FWCI
- 25.77
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.