Exploring the Effectiveness of Flipped Classroom on STEM Student Achievement: A Meta-analysis

Classified as probably not a substantial claim.

Only peer-reviewed journal studies were included, as their results are believed to be of sufficient quality.

“Only peer-reviewed journal studies were included, as their results are believed to be of sufficient quality (Cheng et al., 2019a;Hew & Lo, 2018a;Korpershoek et al., 2016;Lo et al., 2017).”

• cites Cheng et al., 2019a sourceNot checked — no full text of the source
• cites Hew & Lo, 2018a sourceSupported
• cites Korpershoek et al., 2016 sourceNot checked — no full text of the source

Classified as probably not a substantial claim.

Previous reviews found small effect sizes under a random effects model across engineering, mathematics, and health profession education.

“According to some previous reviews (Chen et al., 2018;Hew & Lo, 2018a;Lo et al., 2017), the effect sizes were small under a random effect sizes model across three disciplines (engineering, mathematics, and health profession education).”

• cites Chen et al., 2018 sourcePartly supported

  The Methods (Data synthesis; page unavailable) say the review pooled effect sizes using a random-effects model. The Results (Main and subgroup meta-analyses; page unavailable) report effects for health-science education, including an examination-score effect of 0.53 in medical education, but do not establish that the effects were uniformly small. This source does not separately report findings for engineering or mathematics; the other citations may address those disciplines. (LLM, conf=0.84)

  Experimental — accuracy not yet measured
• cites Hew & Lo, 2018a sourceSupported
• cites Lo et al., 2017Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

May combine two claims.

Hew & Lo (2018a) found that using quizzes at the start of face-to-face courses can help students recall knowledge, identify misconceptions, and improve learning outcomes.

“(Hew & Lo, 2018a) found in their meta-analysis that using quizzes at the start of face-to-face courses can help students recall knowledge and identify possible misconceptions and so improve learning outcomes.”

• cites Hew & Lo, 2018a sourceSupported

May combine two claims.

Most studies in the meta-analysis were conducted within 8 weeks; one lasted half a semester and two lasted only a few days.

“Most of the studies in our meta-analysis were conducted within 8 weeks (Cabi & Emine, 2018;Karabatak & Polat, 2020;Meng et al., 2022;Muzyk et al., 2015;Senousy et al., 2017;Wang et al., 2022;Wilson & Hobbs, 2023), and one studies lasted for half a semester (Chien & Hsieh, 2018) and two studies only lasted for a few days (Arya et al., 2020;Casselman et al., 2020).”

• cites Cabi & Emine, 2018Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Karabatak & Polat, 2020 sourceNot checked — no full text of the source
• cites Meng et al., 2022 sourceNot checked — no full text of the source

May combine two claims.

Publication bias tests showed symmetry: Begg and Mazumdar rank correlation z = 0.74, one-tailed p = 0.4615; Egger's linear regression t = 0.62, one-tailed p = 0.5350; and fail-safe N = 1034 additional missing studies with zero mean effect size would be required to make the overall effect statistically insignificant.

“We conducted two statistical analyses which supported the presence of symmetry: the Begg and Mazumdar rank correlation test (z = 0.74, one-tailed p = 0.4615); and Egger's Linear regression test (t = 0.62, one-tailed p = 0.5350).Both tests indicated that publication bias was not significant.We also conducted a classic fail-safe N test which showed that 1034 additional missing studies with zero mean”

May combine two claims. The quote only partly states this claim.

Flipped classroom courses lasting less than a semester produced better student performance (g = 0.343) than courses lasting a semester or more (g = 0.242), and effect sizes were smaller in studies with longer durations than shorter durations.

“We found that where flipped classroom courses lasted less than a semester, the students performed better (g = 0.343) than those in flipped classroom courses extending for a semester or more (g = 0.242).”

• Also stated in: § 4.1. Flipped classroom promotes student achievement · plain

  “However, it was observed that the effect sizes of the flipped classroom intervention were smaller in studies with longer durations than those with shorter durations.”

May combine two claims.

The flipped classroom effect was stronger when students were engaged in both group activities and individual tasks (g = 0.380) than when they participated only in group activities (g = 0.115) or only in individual tasks (g = 0.278).

“The flipped classroom effect was stronger when students were engaged in both group activities and individual tasks (g = 0.380), compared to when they participated only in group activities (g = 0.115) or only in individual tasks (g = 0.278).”

May combine two claims.

For both shorter and longer durations of study intervention, students in the flipped classroom demonstrated a significant performance advantage over those in the traditional classroom.

“For both shorter and longer durations of study intervention, students in the flipped classroom demonstrated a significant performance advantage over those in the traditional classroom.”

May combine two claims.

The meta-analysis found no evidence of publication bias; the overall mean effect size was not inflated by publication bias.

“with no evidence of publication bias.”

• Also stated in: § 3.4. Analysis of publication bias · boosted

  “Based on the visual inspection of the funnel plot, the statistical analyses, and the fail-safe N test, it can be concluded that the overall mean effect size was not inflated by publication bias.”

May combine two claims.

For studies with a duration of less than a semester, the flipped classroom had a slightly stronger effect (g = 0.343), and students in the flipped classroom significantly outperformed those in the traditional classroom.

“The study duration of less than a semester (k = 13) had a slightly stronger effect size (g = 0.343).When the study duration for flipped classrooms was less than a semester, students in the flipped classroom condition significantly outperformed students in the traditional classroom.”

May combine two claims.

The flipped classroom had a moderate effect on student performance when students engaged in either exercises (g = 0.220) or quizzes (g = 0.275) after class.

“The flipped classroom had a moderate effect on student performance when students engaged in either exercises or quizzes after class (g = 0.220, g = 0.275).”

May combine two claims.

The flipped classroom had a weak effect on student performance when the instructor was the same between the two models (g = 0.227) or when this information was not reported (g = 0.270).

“The flipped classroom had a weak effect on student performance where the instructor was the same between the two models (g = 0.227) or where this information was not reported (g = 0.270).”

May combine two claims.

Studies using different instructors across traditional and flipped classroom conditions yielded a moderate effect size (g = 0.471), based on only three studies.

“Studies using different instructors across the traditional and flipped classroom conditions yielded a moderate effect size (g = 0.471), although this result was based on only three studies.”

May combine two claims.

Our results were inconsistent with previous findings, with the effect size of 0.316 for mathematics being the largest of the disciplines analyzed.

“Our results were inconsistent with these findings, with the effect size of 0.316 for mathematics being the largest of the disciplines analyzed.”

May combine two claims.

There was statistically significant and moderate heterogeneity among the 53 included studies (Q = 108.68, df = 52, p = 0.0001; I2 = 52.2%).

“The heterogeneity analyses showed statistically significant variation among the 53 studies (Q = 108.68,df = 52, p = 0.0001), and the level of observed heterogeneity was moderate (I 2 = 52.2%).”

May combine two claims.

28 of the studies included in the meta-analysis were published after 2020, and only two of them considered the influence of Covid-19 on the delivery mode of in-class activities.

“It is also noticeable that even though 28 of the studies included in our meta-analysis were published after 2020, only two of them considered the influence of Covid-19 on the delivery mode of in-class activities (Ng, 2023;Sezer & Esenay, 2022).”

• cites Ng, 2023Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Sezer & Esenay, 2022 sourceNot checked — no full text of the source

May combine two claims. The quote only partly states this claim.

An effect size of 0.26 is comparable to student-focused interventions and instructional strategies such as individualized instruction, matching style of learning, student-centered teaching, collaborative learning, discovery-based learning, and adjunct aids, and is significantly larger than effect sizes due to feedback strategies and technology approaches such as one-on-one laptops and web-based学习.

“According to analysis standards for influences on student outcomes (Hattie, 2017), an effect size of 0.26 is comparable to those associated with student-focused interventions such as individualized instruction, matching style of learning, and student-centered teaching, as well as those associated with instructional strategies such as collaborative learning, discovery-based learning, and adjunct ai”

• cites Hattie, 2017Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S2 (checkable).

If STEM teaching and learning were not already well structured and scaffolded, the added value of the flipped classroom might be more significant.

“If this were not the case, the added value of the flipped classroom might be more significant.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The literature selection process followed the PRISMA Statement.

“The literature selection process followed the Preferred Reporting of Items for Systematic Reviews and Meta-analyses (PRISMA) Statement (Moher et al., 2009).”

• cites Moher et al., 2009Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Classified as probably not a substantial claim. May combine two claims.

In STEM courses, the flipped classroom's advantages may translate to only modest impacts on student performance partly because teaching and learning is already well structured and incorporates scaffolding for students.

“It is possible that in STEM courses the advantages of the flipped classroom approach translate to only modest impacts on student performance partly because the teaching and learning is already well structured and incorporates scaffolding for students.”

• AssumptionRests on an assumption: STEM teaching and learning is already well structured and incorporates scaffolding for students.

Probably not a substantial claim: may not pass S3 (load-bearing).

The review used standard tests to identify publication bias: funnel plot assessment, Begg and Mazumdar rank correlation, Egger's linear regression, and classic fail-safe N.

“Therefore, the following standard tests were used to identify whether the present review suffered from publication bias, including: 1) assessing the funnel plot (J.Light & B. Pillemer, 1986); 2) computing the Begg & Mazumdar rank correlation (Begg & Mazumdar, 1994); 3) calculating Egger's linear regression (Egger et al., 1997); 4) conducting the classic fail-safe N test (Rosenthal, 1979).”

• cites J.Light & B. Pillemer, 1986 sourceNot checked — no full text of the source
• cites Begg & Mazumdar, 1994 sourceNot checked — no full text of the source
• cites Egger et al., 1997 sourceNot checked — no full text of the source

May combine two claims.

The flipped classroom promotes student achievement in STEM education, with students in flipped classrooms outperforming those in traditional classrooms by 0.26 standard deviations.

“Overall, we found a small positive effect (g = 0.26) of the flipped classroom on assessed student learning outcomes.In other words, students in the flipped classroom outperformed their counterparts in the traditional classroom by 0.26 standard deviations.”

• Also stated in: § Abstract · plain

  “It showed an overall significant effect in favor of flipped learning over traditional learning models for STEM education”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Two authors independently extracted the data, and discrepancies were reviewed, discussed, and resolved.

“Two authors independently extracted the data, and any discrepancies between their extracted data were reviewed, discussed, and resolved.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The articles in this review only covered a few subsets of STEM disciplines, and it is not clear whether the conclusions can be applied to all STEM education contexts.

“Second, although we searched for publications across different databases, the articles in this review only covered a few subsets of STEM disciplines, and it is not clear whether the conclusions can be applied to all STEM education contexts.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The difference in effect when instructors were the same might be attributed to instructors applying flipped classroom skills, such as motivating students towards active learning, even in the traditional classroom, which could moderate the effects.

“This difference might be attributed to the instructors applying the skills utilized in the flipped classroom approach, such as motivating students towards more active learning, even within the traditional classroom setting, which could moderate the effects.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The impact of potential language bias was not taken into consideration.

“The impact of potential language bias was not taken into consideration.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Murphy et al. found that the combined post-class approach was an effective way for students to reflect on knowledge learned in class and to help instructors evaluate student learning performance and reflect on the flipped classroom model as a whole.

“The researchers found that this combined approach to post-class activity was an effective way for students to reflect on knowledge learned in class and to help instructors evaluate student learning performance and reflect on the flipped classroom model as a whole.”

• cites Murphy et al., 2016 sourceNot checked — no full text of the source

Classified as probably not a substantial claim.

Educational goals extend to areas including ethics, lifelong career orientation, civic engagement, and communication.

“They extend to areas including ethics, lifelong career orientation, civic engagement, and communication (National Academies of Sciences & Medicine, 2017).”

• cites National Academies of Sciences & Medicine, 2017Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Classified as probably not a substantial claim.

Feedback is one of the most potent factors affecting learning and achievement.

“Given that feedback is one of the most potent factors affecting learning and achievement (Hattie & Timperley, 2007;Shute, 2008), we suggest that it may play a crucial role in driving the effectiveness of the flipped classroom approach.”

• cites Hattie & Timperley, 2007 sourceNot checked — no full text of the source
• cites Shute, 2008 sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S3 (load-bearing).

This result is consistent with Lo & Hew (2019), which found that the combination of individual tasks and small-group activities produced a larger effect size.

“This result is consistent with the study by (Lo & Hew, 2019) which also found that the combination of individual tasks and small-group activities produced a larger effect size.”

• cites Lo & Hew, 2019 sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S3 (load-bearing).

The selection of reviewed articles was limited to those published in English.

“Although the reviewed articles are from across different continents, the selection was limited to those published in English.”

May combine two claims.

Combining group activities and individual tasks during class and providing quizzes and exercises after class are beneficial components that can improve student achievement in STEM education.

“Combining group activities and individual tasks during class and providing quizzes and exercises after class are beneficial components that can improve student achievement in STEM education, offering promising directions for the future implementation of flipped classroom courses.”

Classified as probably not a substantial claim. May combine two claims.

Students can acquire a deeper understanding of the subject through arguing, collaborating, and knowledge sharing during group activities, which can also enhance their learning when they study individually.

“It may be that students can acquire a deeper understanding of the subject through arguing, collaborating, and knowledge sharing during group activities, which can also enhance their learning when they study individually.”

May combine two claims.

The results of the moderator analysis imply that the flipped classroom can significantly improve students' academic performance in engineering, mathematics, and health profession education.

“The results of the moderator analysis imply that the flipped classroom can significantly improve students' academic performance in the fields of engineering, mathematics, and health profession education.”

Probably not a substantial claim: may not pass S3 (load-bearing).

This finding aligns with Murphy et al. (2016), who required students to do online quizzes and hand in assigned homework problems from the textbook.

“This finding aligns with a study by (Murphy et al., 2016) who required students to do online quizzes and hand in assigned homework problems from the textbook.”

• cites Murphy et al., 2016 sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

The effect may be due to interactions among students during small-group activities, which can have additional cognitive benefits for individual learning performance.

“The effect may be due to interactions among students during small-group activities which can have additional cognitive benefits for their individual learning performance.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Descriptions of interventions and course designs in the reviewed articles varied in quality and completeness, making them difficult to code accurately.

“First, the descriptions of the interventions and course designs in the reviewed articles varied in quality and completeness, making them difficult to code accurately.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Many articles only mentioned small group activities or collaborative team-based learning but did not specify how the activities were organized and how long they lasted.

“Many articles only mentioned the use of small group activities or collaborative team-based learning but did not specify how the activities were organized and how long they lasted.”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

Educational goals encompass more than academic attainment.

“Finally, educational goals encompass more than academic attainment.”

• cites National Academies of Sciences & Medicine, 2017Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.