The DOI matches its Crossref record; 28 of the 28 references it cites were found in a registry or an index.
A meta-analysis of teaching and learning computer programming: Effective instructional approaches and conditions
Open accessAbstract
This meta-analysis maps the evidence on the effectiveness of instructional approaches and conditions for learning computer programming under three study conditions: (a) Studies focusing on the effectiveness of programming interventions per se, (b) studies focusing on the effectiveness of visualization and physicality, and (c) studies focusing on the effectiveness of dominant instructional approaches. Utilizing the data from 139 interventions and 375 effect sizes, we found (a) a strong effect of learning computer programming per se (Hedges’ g‾ = 0.81, 95% CI [0.42, 1.21]), (b) moderate to large effect sizes of visualization (g‾ = 0.44, 95% CI [0.29, 0.58]) and physicality interventions (g‾ = 0.72, 95% CI [0.23, 1.21]), and (c) moderate to large effect sizes for studies focusing on dominant instructional approaches (g‾s = 0.49–1.02). Moderator analyses indicated that the effect sizes differed only marginally between the instructional approaches and conditions—however, collaboration in metacognition instruction, problem solving instruction outside of regular lessons, short-term interventions focusing on physicality, and interventions focusing on visualization through Scratch were especially effective. Our meta-analysis synthesizes the existing research evidence on the effectiveness of computer programming instruction and, ultimately, provides references with which the effects of future studies could be compared.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Wing, J. M. (2006) · Communications of the ACM · doi:10.1145/1118178.1118215 · Cited by 8,102
Outlier and influence diagnostics for meta-analysis
Viechtbauer, W., & Cheung, M. W.-L. (2010) · Research Synthesis Methods · doi:10.1002/jrsm.11 · Cited by 2,949 · cited 1×
“Besides the analysis of publication bias, we identified influential effect sizes using Viechtbauer and Cheung ’ s (2010) diagnostics using the R package ‘metafor ’ .”
MethodThe role of google scholar in evidence reviews and its applicability to grey literature searching
Haddaway, N. R., Collins, A. M., Coughlin, D., & Kirk, S. (2015) · PLOS ONE · doi:10.1371/journal.pone.0138237 · Cited by 2,387
Estimating effect sizes from pretest-posttest-control group designs
Morris, S. B. (2008) · Organizational Research Methods · doi:10.1177/1094428106291059 · Cited by 2,133
Springer, L., Stanne, M. E., & Donovan, S. S. (1999) · Review of Educational Research · doi:10.3102/00346543069001021 · Cited by 1,943 · cited 1×
“Although this evidence base largely supports the effectiveness of collaborative practices, some evidence suggests that the effects may between domains, gender, and the composition of the samples (Springer, Stanne, & Donovan, 1999).”
BackgroundLearning and Teaching Programming: A Review and Discussion
Robins, A., Rountree, J., & Rountree, N. (2003) · Computer Science Education · doi:10.1076/csed.13.2.137.14200 · Cited by 1,736 · cited 1×
“Based on this assumption that programming knowledge and skills can be taught effectively, several instructional approaches have been proposed and evaluated over the last decades — yet, with varying foci and degrees of success (Grover & Pea, 2013; Robins, Rountree, & Rountree, 2003).”
BackgroundBias and Efficiency of Meta-Analytic Variance Estimators in the Random-Effects Model
Viechtbauer, W. (2005) · Journal of Educational and Behavioral Statistics · doi:10.3102/10769986030003261 · Cited by 1,699 · cited 1×
“…for a baseline model with random effects should not only be based on the significance tests of variances and heterogeneity tests only, but relies mainly on the substantive assumptions on whether the effect sizes may or may not vary within or between studies (Cheung, 2015; Viechtbauer, 2005).”
BackgroundSung, Y.-T., Chang, K.-E., & Liu, T.-C. (2016) · Computers & Education · doi:10.1016/j.compedu.2015.11.008 · Cited by 1,574 · cited 1×
“Sung, Chang, and Liu (2016) expected the duration of interventions that examined the effects of integrating mobile devices on students ’ learning to be positive — however, the authors did not find any significant moderation effect of intervention length in their meta-analysis.”
BackgroundP-curve: A key to the file-drawer.
Simonsohn, U., Nelson, L. D., & Simmons, J. P. (2014) · Journal of Experimental Psychology: General · doi:10.1037/a0033242 · Cited by 1,500 · cited 1×
“Third, we examined the p -curve underlying the all intervention effects on computer programming in the data set (Simonsohn, Nelson, & Simmons, 2014).”
MethodDemystifying computational thinking
Shute, V. J., Sun, C., & Asbell-Clarke, J. (2017) · Educational Research Review · doi:10.1016/j.edurev.2017.09.003 · Cited by 1,466
Lye, S. Y., & Koh, J. H. L. (2014) · Computers in Human Behavior · doi:10.1016/j.chb.2014.09.012 · Cited by 1,421 · cited 9×
“In their seminal review, Lye and Koh (2014) argued that computer programming “ exposes students to computational thinking which involves problem-solving using computer science concepts like abstraction and decomposition.”
BackgroundMethodInfluentialCheung, M. W.-L. (2014) · Psychological Methods · doi:10.1037/a0032968 · Cited by 1,342
How Many Studies Do You Need?: A Primer on Statistical Power for Meta-Analysis
Valentine, J. C., Pigott, T. D., & Rothstein, H. R. (2010) · Journal of Educational and Behavioral Statistics · doi:10.3102/1076998609346961 · Cited by 1,294 · cited 1×
“…moderator analyses — moderator effects would have been underpow-ered, and variances and variance explanations may not have been reliably estimated, especially for subgroups of studies containing only one or two effect sizes (e.g., Jackson & Turner, 2017; Valentine, Pigott, & Rothstein, 2010).”
BackgroundDurlak, J. A., Weissberg, R. P., & Pachan, M. (2010) · American Journal of Community Psychology · doi:10.1007/s10464-010-9300-6 · Cited by 1,216 · cited 1×
“One of the reasons for this difference may lie in the presumably enhanced motivation of students enrolling in extracurricular activities, posing a selection effect on the effectiveness of the intervention (e.g., Durlak, Weissberg, & Pachan, 2010).”
BackgroundBernard, R. M., Borokhovski, E., Schmid, R. F., Tamim, R. M., & Abrami, P. C. (2014) · Journal of Computing in Higher Education · doi:10.1007/s12528-013-9077-3 · Cited by 870
Cheung, A. C. K., & Slavin, R. E. (2013) · Educational Research Review · doi:10.1016/j.edurev.2013.01.001 · Cited by 792 · cited 1×
“…relation we found in our meta-analysis may have several reasons, such as the high value of novelty (i.e., when learning programming with a new tool or language) or the higher engagement of students in the intervention tasks in the beginning of the interventions (e.g., Cheung & Slavin, 2013).”
BackgroundHsu, T.-C., Chang, S.-C., & Hung, Y.-T. (2018) · Computers & Education · doi:10.1016/j.compedu.2018.07.004 · Cited by 788 · cited 5×
“…(Lye & Koh, 2014), the benefits of pair programming over individual programming (Umapathy & Ritzhaupt, 2017), and the setup of programming courses, including the effects of blended and project-based learning (Hsu, Chang, & Hung, 2018; Vihavainen, Air-aksinen, & Watson, 2014) — just to name a few.”
BackgroundMethodInfluentialSmall Group and Individual Learning with Technology: A Meta-Analysis
Lou, Y., Abrami, P. C., & d’Apollonia, S. (2001) · Review of Educational Research · doi:10.3102/00346543071003449 · Cited by 721
Macnamara, B. N., Hambrick, D. Z., & Oswald, F. L. (2014) · Psychological Science · doi:10.1177/0956797614535810 · Cited by 656
A Meta-analysis of the Effects of Computer Technology on School Students’ Mathematics Learning
Li, Q., & Ma, X. (2010) · Educational Psychology Review · doi:10.1007/s10648-010-9125-8 · Cited by 558
How Methodological Features Affect Effect Sizes in Education
Cheung, A. C. K., & Slavin, R. E. (2016) · Educational Researcher · doi:10.3102/0013189x16656615 · Cited by 529 · cited 1×
“Examining the moderation effects of study features, we would like to highlight one finding: First, there was no evidence for the differences in effects between pretest-posttest and posttest-only control group designs — in contrast to the general observations noted by Cheung and Slavin (2016).”
ResultA meta-analysis of the impact of technology on learning effectiveness of elementary students
Chauhan, S. (2017) · Computers & Education · doi:10.1016/j.compedu.2016.11.005 · Cited by 395 · cited 3×
“These categories are considered essential in meta-analyses of technology-based interventions (e.g., Bernard, Borokhovski, Schmid, Tamim, & Abrami, 2014; Chauhan, 2017).”
BackgroundInfluentialAdvancing the Science of Collaborative Problem Solving
Graesser, A. C., Fiore, S. M., Greiff, S., Andrews-Todd, J., Foltz, P. W., & Hesse, F. W. (2018) · Psychological Science in the Public Interest · doi:10.1177/1529100618808244 · Cited by 390 · cited 1×
“The latter finding aligns with the tendency of collaborative instruction to be especially effective for skill acquisition in computer-based learning environments as compared to knowledge acquisition (Graesser et al., 2018).”
BackgroundA Meta-Analysis of the Effects of Feedback in Computer-Based Instruction
Azevedo, R., & Bernard, R. M. (1995) · Journal of Educational Computing Research · doi:10.2190/9lmd-3u28-3a0g-ftqt · Cited by 308
Impact of Robotics and Geospatial Technology Interventions on Youth STEM Learning and Attitudes
Nugent, G., Barker, B., Grandgenett, N., & Adamchuk, V. I. (2010) · Journal of Research on Technology in Education · doi:10.1080/15391523.2010.10782557 · Cited by 279
The cognitive benefits of learning computer programming: A meta-analysis of transfer effects.
Scherer, R., Siddiq, F., & Sánchez Viveros, B. (2019) · Journal of Educational Psychology · doi:10.1037/edu0000314 · Cited by 205 · cited 4×
“In their meta-analysis of the transfer effects of learning computer programming on skills other than programming, Scherer et al. (2019) identified a moderate overall effect size, g ¼ 0.47.”
BackgroundMethodInfluentialSelf-citationTsai, Y.-L., & Tsai, C.-C. (2018) · Computers & Education · doi:10.1016/j.compedu.2018.06.020 · Cited by 164 · cited 4×
“Specifically, using the framework for reviewing the effectiveness of educational technology proposed by Chen, Wang, Kirschner, and Tsai (2018), we distinguish between three categories of primary studies to examine three aspects of effectiveness and ultimately map the field of programming…”
BackgroundMethodInfluentialTen quick tips for teaching programming
Brown, N. C. C., & Wilson, G. (2018) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1006023 · Cited by 89 · cited 4×
“Together with Brown and Wilson (2018), we argue that programmers are “ not born but made ”— in other words, we agree that the knowledge and skills involved in programming are not necessarily innate, thus contrasting some beliefs about the nature of programming talent as being innate (Guzdial, 2014).”
BackgroundInfluential
Cited by
Sanders, T., Noetel, M., Parker, P., Del Pozo Cruz, B., Biddle, S., Ronto, R., Hulteen, R., Parker, R., Thomas, G., De Cocker, K., et al. (2023) · Nature Human Behaviour · doi:10.1038/s41562-023-01712-8 · Cited by 148
The cognitive effects of computational thinking: a systematic review and meta-analytic study
Montuori, C., Gambarota, F., Altoé, G., & Arfé, B. (2024) · Computers & Education · doi:10.1016/j.compedu.2023.104961 · Cited by 76
Zhang, L., Jiang, Q., Xiong, W., & Zhao, W. (2025) · Journal of Educational Computing Research · doi:10.1177/07356331251333874 · Cited by 15
Wu, M., & Ouyang, F. (2026) · Educational Psychology Review · doi:10.1007/s10648-026-10211-x
Metadata
- Authors
- Scherer, R., Siddiq, F., & Sánchez Viveros, B.
- Venue
- Computers in Human Behavior
- Publisher
- Elsevier BV
- Published
- 2020-08
- Volume
- 109
- Pages
- 106349
- ISSN
- 0747-5632 (ISSN-L)
- OpenAlex
- W3013431833
- Cited by
- 164
- References
- 28
- FWCI
- 3.46
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.