On the use of flipped classroom across various disciplines: Insights from a second-order meta-analysis
Open accessThe DOI matches its Crossref record; 28 of the 28 references it cites were found in a registry or an index.
Abstract
Flipped classroom has become a popular buzzword in the post-secondary education setting, and it is one of the most visible trends in smart learning environments. Alongside this popularisation comes the view that the flipped classroom is something desirable. Yet, many educators remain divided over whether flipped classroom is really an improvement over traditional approaches. This paper is the first to synthesise all relevant meta-analytic information using a second-order meta-analysis approach on the effectiveness of the flipped classroom on student learning outcomes. By synthesising the findings of multiple primary meta-analyses instead of individual empirical studies, a second-order meta-analysis can more accurately account for publication bias and generate a more robust result. The present study synthesised and analysed the quality of 15 primary meta-analyses that involved 156,722 participants in flipped and non-flipped conditions to provide the most exhaustive test of the flipped classroom approach on its effect on student learning outcomes in higher education to date. The mean random effect size, after trim-and-fill adjustment, was 0.37, p < 0.001 in support of flipped classrooms. To check the accuracy of the second-order meta-analysis results, we performed a study-level meta-analytic validation. We discuss possible contextual and methodological moderators.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Ryan, R. M., & Deci, E. L. (2000) · American Psychologist · doi:10.1037//0003-066x.55.1.68 · Cited by 54,599
Active learning increases student performance in science, engineering, and mathematics
Freeman, S., Eddy, S. L., McDonough, M., Smith, M. K., Okoroafor, N., Jordt, H., & Wenderoth, M. P. (2014) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1319030111 · Cited by 9,787 · cited 7×
“Recent research has found that active learning can significantly increase students’ learning performance (Deslauriers et al., 2019; Freeman et al., 2014).”
BackgroundMethodResultInfluentialThe gamification of learning: a meta-analysis
Sailer, M., & Homner, L. (2020) · Educational Psychology Review · doi:10.1007/s10648-019-09498-w · Cited by 2,594 · cited 2×
“While participants in RCTs are randomly assigned to different groups to avoid selection bias in the experiment (Sailer & Homner, 2020), quasi-experimental studies cannot do so (Wouters et al., 2013).”
BackgroundPublication Bias in Meta‐Analysis: Prevention, Assessment and Adjustments
(2005) · doi:10.1002/0470870168 · Cited by 2,368
A meta-analysis of the cognitive and motivational effects of serious games.
Wouters, P., van Nimwegen, C., van Oostendorp, H., & van der Spek, E. D. (2013) · Journal of Educational Psychology · doi:10.1037/a0031311 · Cited by 1,864
Meta-analysis and the science of research synthesis
Gurevitch, J., Koricheva, J., Nakagawa, S., & Stewart, G. (2018) · Nature · doi:10.1038/nature25753 · Cited by 1,709 · cited 1×
“The primary goal of these first-order meta-analyses (hereafter called primary meta-analyses) is to estimate an overall mean effect from individual empirical studies and to identify possible factors that moderate that effect (Gurevitch et al., 2018).”
BackgroundThe Mass Production of Redundant, Misleading, and Conflicted Systematic Reviews and Meta‐analyses
IOANNIDIS, J. P. A. (2016) · The Milbank Quarterly · doi:10.1111/1468-0009.12210 · Cited by 1,483 · cited 1×
“This is because quality can help minimise the problem of the mass production of ambiguous and misleading meta-analyses (Ioannidis, 2016).”
BackgroundDeslauriers, L., McCarty, L. S., Miller, K., Callaghan, K., & Kestin, G. (2019) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1821936116 · Cited by 1,371
Methodological issues and advances in biological meta-analysis
Nakagawa, S., & Santos, E. S. A. (2012) · Evolutionary Ecology · doi:10.1007/s10682-012-9555-5 · Cited by 1,032
Tamim, R. M., Bernard, R. M., Borokhovski, E., Abrami, P. C., & Schmid, R. F. (2011) · Review of Educational Research · doi:10.3102/0034654310393361 · Cited by 1,029 · cited 3×
“We developed a codebook based on the work conducted by Steenbergen-Hu et al. (2016) and Tamim et al. (2011).”
MethodResultChoosing effect measures and computing estimates of effect
Higgins, J. P., Li, T., & Deeks, J. J. (2019) · Cochrane Handbook for Systematic Reviews of Interventions · doi:10.1002/9781119536604.ch6 · Cited by 977 · cited 2×
“Searching for gray literature such as dissertations or theses is a highly desirable practice recommended by the Cochrane Handbook for Systematic Reviews of Interventions in order to capture as many studies as possible and to minimise the risk of publication bias (Higgins et al., 2019).”
MethodAnatomy of STEM teaching in North American universities
Stains, M., Harshman, J., Barker, M. K., Chasteen, S. V., Cole, R., DeChenne-Peters, S. E., Eagan, M. K., Esson, J. M., Knight, J. K., Laski, F. A., et al. (2018) · Science · doi:10.1126/science.aap8892 · Cited by 772
Peters, J. L., Sutton, A. J., Jones, D. R., Abrams, K. R., & Rushton, L. (2007) · Statistics in Medicine · doi:10.1002/sim.2889 · Cited by 716
A Guide to Conducting a Meta-Analysis with Non-Independent Effect Sizes
Cheung, M. W.-L. (2019) · Neuropsychology Review · doi:10.1007/s11065-019-09415-6 · Cited by 610 · cited 1×
“One of the key assumptions in a primary meta-analysis is that the effect sizes are independent (M. W.-L. Cheung, 2019).”
BackgroundMeta-evaluation of meta-analysis: ten appraisal questions for biologists
Nakagawa, S., Noble, D. W. A., Senior, A. M., & Lagisz, M. (2017) · BMC Biology · doi:10.1186/s12915-017-0357-7 · Cited by 579
How Methodological Features Affect Effect Sizes in Education
Cheung, A. C. K., & Slavin, R. E. (2016) · Educational Researcher · doi:10.3102/0013189x16656615 · Cited by 529 · cited 2×
“Unpublished meta-analyses or empirical works include dissertations and theses (often collectively referred to as “gray literature”) (A. C. K. Cheung & Slavin, 2016, p. 288; Hartling et al., 2017).”
BackgroundResultStrelan, P., Osborn, A., & Palmer, E. (2020) · Educational Research Review · doi:10.1016/j.edurev.2020.100314 · Cited by 526
Cheng, L., Ritzhaupt, A. D., & Antonenko, P. (2019) · Educational Technology Research and Development · doi:10.1007/s11423-018-9633-7 · Cited by 494
Steenbergen-Hu, S., Makel, M. C., & Olszewski-Kubilius, P. (2016) · Review of Educational Research · doi:10.3102/0034654316675417 · Cited by 429
Online Prevention Aimed at Lifestyle Behaviors: A Systematic Review of Reviews
Kohl, L. F., Crutzen, R., & de Vries, N. K. (2013) · Journal of Medical Internet Research · doi:10.2196/jmir.2665 · Cited by 388
Conceptualizing the emerging field of smart learning environments
Spector, J. M. (2014) · Smart Learning Environments · doi:10.1186/s40561-014-0002-7 · Cited by 388 · cited 1×
“In doing so, flipped classrooms aim to be effective, efficient and engaging, thus meeting the three general characteristics expected of smart learning environments (Spector, 2014).”
BackgroundBusch, T., & Friede, G. (2018) · Corporate Social Responsibility and Environmental Management · doi:10.1002/csr.1480 · Cited by 345 · cited 1×
“First, by summarising the findings of more than one primary meta-analysis using a second-order metaanalysis approach, we can generate a more robust generalisable result with a very large sample size (Busch & Friede, 2018).”
MethodKung, J. (2010) · The Open Dentistry Journal · doi:10.2174/1874210601004020084 · Cited by 329 · cited 5×
“R-AMSTAR was developed based on the widely used AMSTAR instrument (Kung et al., 2010).”
BackgroundMethodInfluentialHigher education dominance and siloed knowledge: a systematic review of flipped classroom research
Lundin, M., Bergviken Rensfeldt, A., Hillman, T., Lantz-Andersson, A., & Peterson, L. (2018) · International Journal of Educational Technology in Higher Education · doi:10.1186/s41239-018-0101-6 · Cited by 227 · cited 1×
“The flipped classroom approach has been growing fast, particularly in higher education settings (Lundin et al., 2018).”
BackgroundCooper, H., & Koenka, A. C. (2012) · American Psychologist · doi:10.1037/a0027119 · Cited by 196
Borenstein, M. (2005) · Publication Bias in Meta‐Analysis · doi:10.1002/0470870168.ch11 · Cited by 178 · cited 1×
“Further tests, such as the classic fail-safe N, as well as statistical tests (e.g., Begg and Mazumdar rank correlation) which can “quantify the amount of bias” (Borenstein, 2005, p. 195) should be conducted.”
MethodLo, C. K., & Hew, K. F. (2019) · Journal of Engineering Education · doi:10.1002/jee.20293 · Cited by 174 · cited 3×
“Multiple exposure to course materials can help improve student understanding of the lesson (Lo & Hew, 2019; Yelamarthi & Drake, 2015; Yelamarthi et al., 2016).”
BackgroundSelf-citationOutcome-Reporting Bias in Education Research
Pigott, T. D., Valentine, J. C., Polanin, J. R., Williams, R. T., & Canada, D. D. (2013) · Educational Researcher · doi:10.3102/0013189x13507104 · Cited by 127
Cited by
Schmid, R. F., Borokhovski, E., Bernard, R. M., Pickup, D. I., & Abrami, P. C. (2023) · Computers and Education Open · doi:10.1016/j.caeo.2023.100142 · Cited by 70
The effects of gamified flipped classroom on student learning: evidence from a meta-analysis
Yu, Q., & Yu, K. (2024) · Interactive Learning Environments · doi:10.1080/10494820.2023.2209791 · Cited by 26
Metadata
- Authors
- Hew, K. F., Bai, S., Huang, W., Dawson, P., Du, J., Huang, G., Jia, C., & Thankrit, K.
- Venue
- Australasian Journal of Educational Technology
- Published
- 2021-05-10
- Volume
- 37
- Issue
- 2
- Pages
- 132-151
- ISSN
- 1449-3098 (ISSN-L)
- 1449-5554
- OpenAlex
- W3160044404
- Cited by
- 46
- References
- 28
- FWCI
- 11.43
- Type
- journal article
- Language
- en
- Open access
- diamond
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.