The DOI matches its Crossref record; 73 of the 75 references it cites were found in a registry or an index.
College students' cognitive learning outcomes in technology-enabled active learning environments: a meta-analysis of the empirical literature
Abstract
Technology-enabled active learning environments (TE-ALEs) have attracted considerable research interest, particularly in higher education. However, research shows inconsistent results describing the influence of TE-ALEs toward students’ cognitive learning outcomes. This study was designed to identify high-quality empirical research examining college students’ cognitive learning outcomes and to utilize meta-analysis to determine the overall effectiveness of TE-ALEs. A systematic literature search identified 31 high-quality peer-reviewed journal articles that met the inclusion criteria. Meta-analysis showed that the calculated effect size of TE-ALEs more positively influenced students’ cognitive learning than traditional lecture-based environments. Moderator variable analysis suggested that social context, study design, and sample size were significant factors that influence the effectiveness of TE-ALE. TE-ALEs were found more effective when instructors employed individualized learning contexts as well as when bias was reduced in randomized controlled trials. TE-ALEs were also found to be more effective in small courses rather than in large courses.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Meta-analytic findings indicate that although technology-enabled learning environments can support cognitive outcomes, effect sizes vary widely across instructional designs and learning contexts, highlighting the lack of consistent conclusions regarding how and when digital tools enhance learning (Shi et al., 2020; Hu et al., 2025).”
From Cognitive offloading through digital tools and its relationship with critical thinking, task persistence, and learning depth (2026) · cites this paper as Shi et al., 2020
In the citing paper
“In meta-analyses that compared studies that were shorter and longer than a semester (Cheng et al., 2019b;Shi, Ma, et al., 2020) and compared studies that lasted 1-4, 5-8, 9-12, 13-16 weeks, and more than 16 weeks (Karagöl & Esen, 2019;Tutal & Yazar, 2021), the effects gradually decreased as the durations increased.”
From Exploring the Effectiveness of Flipped Classroom on STEM Student Achievement: A Meta-analysis (2024) · cites this paper as Shi, Ma, et al., 2020
In the citing paper
“This result was consistent with the findings of previous metaanalyses.”
From Exploring the Effectiveness of Flipped Classroom on STEM Student Achievement: A Meta-analysis (2024) · cites this paper as Shi, Ma, et al., 2020
Citations
Cites
Measuring inconsistency in meta-analyses
Higgins, J. P. T., Thompson, S. G., Deeks, J. J., & Altman, D. G. (2003) · BMJ · doi:10.1136/bmj.327.7414.557 · Cited by 63,139 · cited 1×
“The presence of heterogeneity (i.e., the degree of inconsistency in the studies’ results) was detected by the I 2 test (Higgins, Thompson, Deeks, & Altman, 2003).”
MethodActive learning increases student performance in science, engineering, and mathematics
Freeman, S., Eddy, S. L., McDonough, M., Smith, M. K., Okoroafor, N., Jordt, H., & Wenderoth, M. P. (2014) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1319030111 · Cited by 9,787 · cited 4×
“For instance, Freeman et al. (2014) meta-analyzed 225 studies across different scientific disciplines, study designs, and class sizes to determine whether active learning improves examination scores when compared with traditional lecture-based instruction.”
BackgroundInfluentialThe file drawer problem and tolerance for null results.
Rosenthal, R. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.3.638 · Cited by 9,068 · cited 2×
“…the fail-safe number (Nfs), which estimates the number of unretrieved studies averaging null results needed to bring the overall combined effect size at a nonsignificant level (Rosenthal, 1979), was calculated at the significant level of .05 by using the formula: Nfs 0 : 05 ¼ P Z 1 : 645 (cid:2) (cid”
BackgroundMethodDoes Active Learning Work? A Review of the Research
Prince, M. (2004) · Journal of Engineering Education · doi:10.1002/j.2168-9830.2004.tb00809.x · Cited by 7,334 · cited 7×
“Active learning is often contrasted with traditional lecture-based instructional methods, where students passively receive information from the instructor (Prince, 2004).”
BackgroundMethodInfluentialHake, R. R. (1998) · American Journal of Physics · doi:10.1119/1.18809 · Cited by 6,072 · cited 1×
“Active learning methods are important even in large environments, as reflected in the study conducted by Hake (1998).”
BackgroundPrimary, Secondary, and Meta-Analysis of Research
GLASS, G. V. (1976) · Educational Researcher · doi:10.3102/0013189x005010003 · Cited by 4,890 · cited 1×
“Meta-analysis is a quantitative statistical method for synthesizing and integrating descriptive statistics reported in multiple relevant published and unpublished primary research testing the same conceptual question and hypothesis (Glass, 1976; Hedges & Olkin, 1985).”
MethodMeta-Analytic Procedures for Social Research
Rosenthal, R. (1991) · doi:10.4135/9781412984997 · Cited by 4,635 · cited 1×
“The large the Nfs is, especially when Nfs is significantly exceeded the critical value 5 k + 10 (Rosenthal, 1991), then it can be concluded that the mean effect size can be considered robust, which suggest that there is no indication of publication bias (Hoeve et al., 2012).”
BackgroundThe Journal of College Science Teaching
(1971) · Journal of Chemical Education · doi:10.1021/ed048p284 · Cited by 3,332
Springer, L., Stanne, M. E., & Donovan, S. S. (1999) · Review of Educational Research · doi:10.3102/00346543069001021 · Cited by 1,943 · cited 3×
“While several meta-analyses, especially of STEM (science, technology, engineering, and mathematics) courses (e.g., Freeman et al., 2014; Springer, Stanne, & Donovan, 1999), explored the effectiveness of active learning pedagogy on student cognitive learning in undergraduate education, little…”
BackgroundBernard, R. M., Abrami, P. C., Lou, Y., Borokhovski, E., Wade, A., Wozney, L., Wallet, P. A., Fiset, M., & Huang, B. (2004) · Review of Educational Research · doi:10.3102/00346543074003379 · Cited by 1,761
Research works printed but not linked: 10
- *Aşıksoy G., 2016, Eurasia Journal of Mathematics, Science & Technology Education, 12, 1589
- Cochrane Collaboration. (2014). Review manager (Version 5.3) [Computer software]. Retrieved from https://community.cochrane.org/help/tools-and-software/revman-5/revman-5-download.
- *Cotner S., 2013, Journal of College Science Teaching, 42, 82
- *Entezari M., 2016, International Journal of Higher Education, 5, 222
- *Heuett W. J., 2017, Problems, Resources, and Issues in Mathematics Undergraduate Studies (Primus), 27, 889
- Kovalchick A., 2004, Education and technology: An encyclopedia (Vol. 1)
- Mazur E., 2004, Interactive teaching: Promoting better learning using peer instruction and just-in-time teaching
- *Shieh R. S., 2011, Australasian Journal of Educational Technology, 27, 1082
- *Shieh R. S., 2010, The Asia Pacific Education Researcher, 19, 401
- *Wilson L. V. E., 2012, Southern Journal of Business & Ethics, 4, 101
Cited by
Sailer, M., Maier, R., Berger, S., Kastorff, T., & Stegmann, K. (2024) · Learning and Individual Differences · doi:10.1016/j.lindif.2024.102446 · Cited by 192
Razak, A. A., Ramdan, M. R., Mahjom, N., Zabit, M. N. M., Muhammad, F., Hussin, M. Y. M., & Abdullah, N. L. (2022) · International Journal of Learning Teaching and Educational Research · doi:10.26803/ijlter.21.2.19 · Cited by 176
“More evidence on teaching and learning processes shows that the use of active learning methods and technology is highly useful in adding dynamism to the setting (Shi et al., 2019; Coorey, 2016).”
BackgroundZheng, L., Long, M., Zhong, L., & Gyasi, J. F. (2022) · Education and Information Technologies · doi:10.1007/s10639-022-11092-7 · Cited by 168
Effects of mind mapping-based instruction on student cognitive learning outcomes: a meta-analysis
Shi, Y., Yang, H., Yi, D., & Zeng, Y. (2022) · Asia Pacific Education Review · doi:10.1007/s12564-022-09746-9 · Cited by 142
Self-citationAltay, S., Hacquin, A.-S., Chevallier, C., & Mercier, H. (2021) · Journal of Experimental Psychology Applied · doi:10.1037/xap0000400 · Cited by 116
Metadata
- Authors
- Shi, Y., Yang, H., MacLeod, J., Zhang, J., & Yang, H. H.
- Affiliations
- Central China Normal University (CN)
- D'Youville University (US)
- Corresponding author
- Yang, Harrison Hao
- Venue
- Journal of Educational Computing Research
- Publisher
- SAGE Publications
- Published
- 2019-10-11
- Volume
- 58
- Issue
- 4
- Pages
- 791-817
- ISSN
- 0735-6331 (ISSN-L)
- 1541-4140
- OpenAlex
- W2979391008
- Cited by
- 62
- References
- 75
- Citations by year
- 2026: 7 · 2025: 5 · 2024: 10 · 2023: 8 · 2022: 17 · 2021: 7 · 2020: 8
- FWCI
- 2.29
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
- Locations
- Journal of Educational Computing Research (publishedVersion)
- Funders
- This study was supported by the Ministry of Education of the Humanities and Social Science project (18YJC880074)
- Topics
- Social Sciences › Education › Innovative Teaching Methods
- Social Sciences › Psychology › Developmental and Educational Psychology › Innovative Teaching and Learning Methods
- Social Sciences › Education › Online and Blended Learning
- Keywords
- cognitive learning outcomes, university students, higher education, active learning, lecture method, effect size, personalized learning, randomized controlled trials, social environment
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: College students' cognitive learning outcomes in technology-enabled active learning environments: a meta-analysis of the empirical literature. https://app.cochat.ai/papers/10.1177/0735633119881477
- The paper
- Shi, Y., Yang, H., MacLeod, J., Zhang, J., & Yang, H. H. (2020). College students' cognitive learning outcomes in technology-enabled active learning environments: a meta-analysis of the empirical literature. Journal of Educational Computing Research. https://doi.org/10.1177/0735633119881477
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.