Evaluating the Effectiveness of Parsons Problems for Block-based Programming
Open accessAbstract
Parsons problems are program puzzles, where students piece together code fragments to construct a program. Similar to block-based programming environments, Parsons problems eliminate the need to learn syntax. Parsons problems have been shown to improve learning efficiency when compared to writing code or fixing incorrect code in lab studies, or as part of a larger curriculum. In this study, we directly compared Parsons problems with block-based programming assignments in classroom settings. We hypothesized that Parsons problems would improve students' programming efficiency on the lab assignments where they were used, without impacting performance on the subsequent, related homework or the later programming project. Our results confirmed our hypothesis, showing that on average Parsons problems took students about half as much time to complete compared to equivalent programming problems. At the same time, we found no evidence to suggest that students performed worse on subsequent assignments, as measured by performance and time on task. The results indicate that the effectiveness of Parsons problems is not simply based on helping students avoid syntax errors. We believe this is because Parsons problems dramatically reduce the programming solution space, letting students focus on solving the problem rather than having to solve the combined problem of devising a solution, searching for needed components, and composing them together.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing
Benjamini, Y., & Hochberg, Y. (1995) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1995.tb02031.x · Cited by 109,482 · see citing works
Cognitive Architecture and Instructional Design
Sweller, J., van Merrienboer, J. J. G., & Paas, F. G. W. C. (1998) · Educational Psychology Review · doi:10.1023/a:1022193728205 · Cited by 6,023 · see citing works
Multiple Comparisons Using Rank Sums
Dunn, O. J. (1964) · Technometrics · doi:10.1080/00401706.1964.10490181 · Cited by 5,168 · see citing works
Nine Ways to Reduce Cognitive Load in Multimedia Learning
Mayer, R. E., & Moreno, R. (2003) · Educational Psychologist · doi:10.1207/s15326985ep3801_6 · Cited by 4,207 · see citing works
Element Interactivity and Intrinsic, Extraneous, and Germane Cognitive Load
Sweller, J. (2010) · Educational Psychology Review · doi:10.1007/s10648-010-9128-5 · Cited by 2,020 · see citing works
Cox, T. J., & Cox, M. (2000) · Chapman & Hall/CRC eBooks · doi:10.1201/9780367801700 · Cited by 752 · see citing works
Comparing Block-Based and Text-Based Programming in High School Computer Science Classrooms
Weintrop, D., & Wilensky, U. J. (2017) · ACM Transactions on Computing Education · doi:10.1145/3089799 · Cited by 338 · see citing works
Strategies for Computer-Based Programming Instruction: Program Completion vs. Program Generation
Van Merriënboer, J. J. G., & De Croock, M. B. M. (1992) · Journal of Educational Computing Research · doi:10.2190/mjdx-9pp4-kfmt-09pm · Cited by 207 · see citing works
Evaluating a new exam question
Denny, P. C., Luxton-Reilly, A., & Simon, B. (2008) · doi:10.1145/1404520.1404532 · Cited by 203 · see citing works
Comparing Textual and Block Interfaces in a Novice Programming Environment
Price, T., & Barnes, T. M. (2015) · doi:10.1145/2787622.2787712 · Cited by 196 · see citing works
Self-citationParson's programming puzzles: a fun and effective learning tool for first programming courses
Parsons, D., & Haden, P. (2006) · Australasian Computing Education Conference · Cited by 195 · see citing works
Price, T., Dong, Y., & Lipovac, D. (2017) · ACM Technical Symposium on Computer Science Education (SIGCSE) · doi:10.1145/3017680.3017762 · Cited by 140 · see citing works
Self-citationSubgoals Help Students Solve Parsons Problems
Morrison, B. B., Margulieux, L. E., Ericson, B. J., & Guzdial, M. (2016) · ACM Technical Symposium on Computer Science Education (SIGCSE) · doi:10.1145/2839509.2844617 · Cited by 126 · see citing works
Solving parsons problems versus fixing and writing code
Ericson, B. J., Margulieux, L. E., & Rick, J. (2017) · Proceedings of the 17th Koli Calling International Conference on Computing Education Research · doi:10.1145/3141880.3141895 · Cited by 121 · see citing works
Analysis of Interactive Features Designed to Enhance Learning in an Ebook
Ericson, B. J., Guzdial, M., & Morrison, B. B. (2015) · ACM Conference on International Computing Education Research (ICER) · doi:10.1145/2787622.2787731 · Cited by 102 · see citing works
How do students solve parsons programming problems?
Helminen, J., Ihantola, P., Karavirta, V., & Malmi, L. (2012) · ACM Conference on International Computing Education Research (ICER) · doi:10.1145/2361276.2361300 · Cited by 99 · see citing works
Identifying Design Principles for CS Teacher Ebooks through Design-Based Research
Ericson, B. J., Rogers, K., Parker, M. C., Morrison, B. B., & Guzdial, M. (2016) · ACM Conference on International Computing Education Research (ICER) · doi:10.1145/2960310.2960335 · Cited by 78 · see citing works
Evaluating the Efficiency and Effectiveness of Adaptive Parsons Problems
Ericson, B. J., Foley, J. D., & Rick, J. (2018) · Proceedings of the 2018 ACM Conference on International Computing Education Research · doi:10.1145/3230977.3231000 · Cited by 70 · see citing works
Factors Influencing Students' Help-Seeking Behavior while Programming with Human and Computer Tutors
Price, T. W., Liu, Z., Cateté, V., & Barnes, T. (2017) · Proceedings of the 2017 ACM Conference on International Computing Education Research · doi:10.1145/3105726.3106179 · Cited by 61 · see citing works
Self-citationDistractors in Parsons Problems Decrease Learning Efficiency for Young Novice Programmers
Harms, K. J., Chen, J., & Kelleher, C. L. (2016) · ACM Conference on International Computing Education Research (ICER) · doi:10.1145/2960310.2960314 · Cited by 60 · see citing works
Two-Dimensional Parson’s Puzzles: The Concept, Tools, and First Observations
Ihantola, P., & Karavirta, V. (2011) · Journal of Information Technology Education Innovations in Practice · doi:10.28945/1394 · Cited by 56 · see citing works
Automating Hint Generation with Solution Space Path Construction
Rivers, K., & Koedinger, K. R. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-07221-0_41 · Cited by 56 · see citing works
Evaluation of a Data-Driven Feedback Algorithm for Open-Ended Programming.
Price, T., Zhi, R., & Barnes, T. M. (2017) · Educational Data Mining · Cited by 43 · see citing works
Self-citationGenerating Data-Driven Hints for Open-Ended Programming.
Price, T., Dong, Y., & Barnes, T. M. (2016) · Educational Data Mining · Cited by 41 · see citing works
Self-citationEnabling independent learning of programming concepts through programming completion puzzles
Harms, K. J., Rowlett, N., & Kelleher, C. L. (2015) · IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) · doi:10.1109/vlhcc.2015.7357226 · Cited by 35 · see citing works
How to study programming on mobile touch devices
Ihantola, P., Helminen, J., & Karavirta, V. (2013) · Koli Calling International Conference on Computing Education Research · doi:10.1145/2526968.2526974 · Cited by 27 · see citing works
Cheng, N., & Harrington, B. (2017) · doi:10.1145/3017680.3017704 · Cited by 26 · see citing works
Garner, S. (2007) · Informing Science and IT Education Conference · doi:10.28945/3127 · Cited by 21 · see citing works
How Do Students Solve Parsons Programming Problems? -- Execution-Based vs. Line-Based Feedback
Helminen, J., Ihantola, P., Karavirta, V., et al. (2013) · doi:10.1109/latice.2013.26 · Cited by 19 · see citing works
Garner, S. (2007) · Issues in Informing Science and Information Technology · doi:10.28945/966 · Cited by 13 · see citing works
Towards Understanding Successful Novice Example Use in Blocks-Based Programming
Ichinco, M., Harms, K., & Kelleher, C. (2017) · Journal of Visual Languages and Sentient Systems · doi:10.18293/vlss2017-012 · Cited by 10 · see citing works
Flagged first, then by citations
Research works printed but not linked: 3
- [10]10. Geela Venise Firmalo Fabic, Antonija Mitrovic, and Kourosh Neshatian. 2017. A Comparison of Different Types of Learning Activities in a Mobile Python Tutor. Proceedings of the 25th International Conference on Computers in Education (2017), 604-613.
- [24]24. Thomas W Price, Yihuan Dong, and Dragan Lipovac. 2017. iSnap: towards intelligent tutoring in novice programming environments. In Proceedings of the 2017 ACM SIGCSE Technical Symposium on Computer Science Education. ACM, 483-488.
- [32]32. Rui Zhi, Thomas W. Price, Nicholas Lytle, Yihuan Dong, and Tiffany Barnes. 2018. Reducing the State Space of Programming Problems through Data-Driven Feature Detection. In EDM Workshop.
Cited by
Sorted by citations
Studying the effect of AI Code Generators on Supporting Novice Learners in Introductory Programming
Kazemitabaar, M., Chow, J., Ma, C. K. T., Ericson, B. J., Weintrop, D., & Grossman, T. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580919 · Cited by 366 · see citing works
“Learner’s typically complete Parsons problems significantly more quickly than fixing or writing code and have similar learning gains [24, 25, 94].”
Cognitive Load Theory in Computing Education Research: A Review
Duran, R., Zavgorodniaia, A., & Sorva, J. (2022) · ACM Transactions on Computing Education · doi:10.1145/3483843 · Cited by 92 · see citing works
Haynes-Magyar, C. (2024) · Proceedings of the 55th ACM Technical Symposium on Computer Science Education V. 1 · doi:10.1145/3626252.3630898 · Cited by 9 · see citing works
- Authors
- Zhi, R., Chi, M., Barnes, T., & Price, T. W.
- Affiliations
- North Carolina State University (US)
- Venue
- Proceedings of the 2019 ACM Conference on International Computing Education Research
- Publisher
- ACM
- Published
- 2019-07-30
- Pages
- 51-59
- OpenAlex
- W2964982419
- Semantic Scholar
- 8deb8968e738230a0ce515503040fe78c540b768
- Cited by
- 42
- References
- 31
- Citations by year
- 2026: 3 · 2025: 5 · 2024: 6 · 2023: 7 · 2022: 10 · 2021: 8 · 2020: 3
- FWCI
- 6.08
- Type
- conference paper
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- ACM Conference on International Computing Education Research (ICER) (publishedVersion)
- Funders
- National Science Foundation (1623470)
- Topics
- Physical Sciences › Computer Science › Computer Science Applications › Teaching and Learning Programming
- Physical Sciences › Computer Science › Artificial Intelligence › Educational Robotics and Engineering
- Social Sciences › Education › Problem and Project Based Learning
- Keywords
- Parsons problems, block-based programming, programming education, programming efficiency, syntactic errors, solution space reduction, programming exercises, homework performance
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
