Solving parsons problems versus fixing and writing code
The DOI matches its Crossref record; 56 of the 56 references it cites were found in a registry or an index.
Abstract
Prior research has shown that Parsons problems are an engaging type of code completion problem that can be used to teach syntactic and semantic language constructs. They can also be used in summative assessments to reduce marking time and grading variability compared to code writing problems. In a Parsons problem the correct code is provided, but is broken into mixed-up code blocks that must be assembled in the correct order. Two-dimensional Parsons problems also require the code blocks to be indented correctly. Parsons problems can contain extra code blocks, called distractors, which are not needed in a correct solution. We present a study that compared the efficiency, effectiveness, and cognitive load of learning from solving two-dimensional Parsons problems with distractors, versus fixing code with the same errors as the distractors, versus writing the equivalent code. We found that solving two-dimensional Parsons problems with distractors took significantly less time than fixing code with errors or than writing the equivalent code. Additionally, there was no statistically significant difference in the learning performance, or in student retention of the knowledge one week later.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Recently, Ericson et al. found that interleaving WEs with Parsons problems can improve programming learning efficiency compared to WEs paired with writing code or fixing code [3].”
From Exploring the Impact of Worked Examples in a Novice Programming Environment (2019) · cites this paper as [3]
Citations
Cites
Cognitive Load During Problem Solving: Effects on Learning
Sweller, J. (1988) · Cognitive Science · doi:10.1016/0364-0213(88)90023-7 · Cited by 11,048
Cognitive Load Theory and Instructional Design: Recent Developments
Paas, F., Renkl, A., & Sweller, J. (2003) · Educational Psychologist · doi:10.1207/s15326985ep3801_1 · Cited by 3,041
(2000) · Cited by 2,779
Ericsson, K. A. (2006) · The Cambridge Handbook of Expertise and Expert Performance · doi:10.1017/cbo9780511816796.038 · Cited by 1,629
Active‐Constructive‐Interactive: A Conceptual Framework for Differentiating Learning Activities
Chi, M. T. H. (2009) · Topics in Cognitive Science · doi:10.1111/j.1756-8765.2008.01005.x · Cited by 1,597
Statistics for behavioral sciences
Gravetter, F. J., & Wallnau, L. B. (1985) · Cited by 1,104
Development of an instrument for measuring different types of cognitive load
Leppink, J., Paas, F., Van der Vleuten, C. P. M., Van Gog, T., & Van Merriënboer, J. J. G. (2013) · Behavior Research Methods · doi:10.3758/s13428-013-0334-1 · Cited by 1,066
Learning from Examples: Instructional Principles from the Worked Examples Research
Atkinson, R. K., Derry, S. J., Renkl, A., & Wortham, D. (2000) · Review of Educational Research · doi:10.2307/1170661 · Cited by 1,029
Taking the Load Off a Learner's Mind: Instructional Design for Complex Learning
van Merrienboer, J. J. G., Kirschner, P. A., & Kester, L. (2003) · Educational Psychologist · doi:10.1207/s15326985ep3801_2 · Cited by 857
(2011) · Cited by 725
Research works printed but not linked: 6
- 10.3102/00346543070002181 · doi:10.3102/00346543070002181 · Paper page
- John D. Bransford , Ann L. Brown and Rodney R . Cocking How People Learn. NATIONAL ACADEMY PRESS , 2000 . John D. Bransford, Ann L. Brown and Rodney R. Cocking How People Learn. NATIONAL ACADEMY PRESS, 2000.
- 10.1037/0096-3445.127.4.355 · doi:10.1037/0096-3445.127.4.355 · Paper page
- Barbara Ericson Java Review Book for the AP CS A exam https://runestone.academy/runestone/static/JavaReview/index.html Barbara Ericson Java Review Book for the AP CS A exam https://runestone.academy/runestone/static/JavaReview/index.html
- William MK Trochim and James P Donnelly Research methods knowledge base ( 3 rd ed). Atomic Dog , Cincinnati, OH , 2006 . William MK Trochim and James P Donnelly Research methods knowledge base (3rd ed). Atomic Dog, Cincinnati, OH, 2006.
- Barry J. Wadsworth Piaget's Theory of Cognitive and Affective Development - Fourth Edition. Longman , New York 1989 . Barry J. Wadsworth Piaget's Theory of Cognitive and Affective Development - Fourth Edition. Longman, New York 1989.
Cited by
Items 101–146 of 146
A C Language Learning Platform Based on Parsons Problems
Sheng, Y., Li, B., Wu, Z., Zhong, P., & Duan, G. (2023) · Communications in computer and information science · doi:10.1007/978-981-99-2446-2_49 · Cited by 3
Toward Ability-Based Design for Novice Programmers with Learning (Dis)abilities
Haynes-Magyar, C. (2020) · ACM Conference on International Computing Education Research (ICER) · doi:10.1145/3372782.3407104 · Cited by 3
Do Students Use Semantics When Solving Parsons Puzzles? – A Log-Based Investigation
Kumar, A. N. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-80421-3_49 · Cited by 3
Gopal, B., & Cooper, S. (2021) · IEEE Frontiers in Education Conference (FIE) · doi:10.1109/fie49875.2021.9637353 · Cited by 3
JsStories: Improving Social Inclusion in Computer Science Education Through Interactive StoriesPreprint
Ghailani, I. G., Malaise, Y., & Signer, B. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2504.04006 · Cited by 3
Curriculum for a Comprehensive Statewide In-Service CS Teacher Training Program
Diesburg, S., Schafer, J. B., & Morrison, B. B. (2025) · ACM Technical Symposium on Computer Science Education (SIGCSE) · doi:10.1145/3641554.3701882 · Cited by 2
Quantifying Novice Behavior, Experience, and Mental Effort in Code Puzzle Pathways
Allen, J., & Kelleher, C. L. (2021) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3411763.3451752 · Cited by 2
Bugs as Features: Describing Patterns in Student Code through a Classification of Bugs
Malysheva, Y., & Kelleher, C. L. (2020) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3334480.3383065 · Cited by 2
Declercq, C., Lehuen, J., Leroux, P., Renault, V., & Séjourné, A. (2022) · HAL · Cited by 2
Proof Blocks: Autogradeable Scaffolding Activities for Learning to Write Proofs.Preprint
Poulsen, S., Viswanathan, M., Herman, G. L., & West, M. (2021) · arXiv (Cornell University) · Cited by 2
Neutens, T. (2022) · Ghent University Academic Bibliography (Ghent University) · Cited by 2
Yang, S. S. (2025) · Illinois Digital Environment for Access to Learning and Scholarship (University of Illinois at Urbana-Champaign) · Cited by 2
Breaking the Textbook Paradigm: Increasing Access by Removing Words
Deitrick, E., Lyons, M., & Ball, J. (2024) · ASEE Annual Conference & Exposition · doi:10.18260/1-2--48408 · Cited by 1
A Randomized Controlled Trial of Syntax Exercises in an Introductory Python Course
Isenegger, K., Otmani, S. E., Edwards, J., & Lewis, C. M. (2025) · ACM Conference on International Computing Education Research (ICER) · doi:10.1145/3702652.3744230 · Cited by 1
The Effects of Starter Template Scaffoldings on Coding Anxiety in Introductory Programming Courses
Bolloju, N., Malapati, A., & Upadhyay, P. (2026) · ACM Transactions on Computing Education · doi:10.1145/3796506 · Cited by 2
Comparative Analysis of Students’ Perceptions of Programming Puzzles: Parson’s and Wordle-Like
Varnavsky, A. N. (2026) · IEEE Access · doi:10.1109/access.2026.3663888 · Cited by 1
An Innovative Approach to Parsons Problems for Teaching and Learning Functional Programming
Bell, J., Deschamps, A., Kapoor, E., Karki, S., Kim, J., González, N. M., Pitchford, W., Sturua, E., Wade, C., & Rebelsky, S. A. (2026) · ACM Technical Symposium on Computer Science Education (SIGCSE) · doi:10.1145/3770762.3772625 · Cited by 1
Pădurean, V.-A., Riegel, K., Barbre, G., Blake, M., Denny, P. C., Gotovos, A., Leinonen, J., MacNeil, S., Prather, J. E., & Singla, A. K. (2026) · ACM Conference on International Computing Education Research (ICER) · doi:10.1145/3765964.3811639 · Cited by 1
Design and Evaluation of Instructional Supports for Novice Programming Environments.
Zhi, R. (2019) · NCSU Libraries Repository (North Carolina State University Libraries) · Cited by 1
Gaspar, A., Bari, A. T. M. G., Vitel, D., Tan, K. C., Albert, J. L., & Wiegand, R. P. (2020) · ASEE Annual Conference & Exposition · doi:10.18260/1-2--32383 · Cited by 1
Grover, S., Carmichael, B., & Venkatasubramaniam, S. (2022) · ACM Technical Symposium on Computer Science Education (SIGCSE) · doi:10.1145/3478432.3499080 · Cited by 1
Actes du colloque Didapro 10 sur la Didactique de l'informatique et des STIC
Goletti, O., & Mens, K. (2024) · HAL · Cited by 1
Amiri, S. M. H., & Islam, M. M. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5240568 · Cited by 1
Choose Your Own Solution: Supporting Optional Blocks in Block Ordering ProblemsPreprint
Oakeson, S., Smith, D. H., Winder, J., & Poulsen, S. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2510.11999 · Cited by 1
Efficient Generation of Parsons Problems for Digital Programming Exams in Inspera
Jørgensen, J. W. H., & Kvannli, S. (2019) · NVA (Nasjonalt vitenarkiv) · Cited by 1
Generating Teacher Feedback through Parsons Problems
Scholten, L. (2020) · Research Repository (Delft University of Technology)
Improving Pedagogy of Computer Programming Through Orthogonal Skill Isolation
Kruger, D., & Baxter, G. (2024) · doi:10.18260/1-2--36049
Dong, Y. (2021) · NCSU Libraries Repository (North Carolina State University Libraries)
ScaffoldSQL: Using Parson’s Problems to Support Database Pedagogy
Borchert, O., & Walia, G. (2022) · Hawaii International Conference on System Sciences (HICSS) · doi:10.24251/hicss.2022.127
Using Markov Matrix to Analyze Students' Strategies for Solving Parsons Puzzles
Kumar, A. N. (2023) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.8115678
Integrating Parsons problems in a CS1 programming course autograder
Lengelé, C. (2024) · HAL
Subgoal Learning Integration in a CS1 Course
Goletti, O., & STIC, D. 1. S. L. D. D. L. E. D. (2024) · HAL
Goletti, O., Mens, K., & Hermans, F. (2024) · ACM SIGPLAN International Symposium on SPLASH-E (SPLASH-E) · doi:10.1145/3689493.3689986
Scheirer, C., Castro, F. E. V., & DesPortes, K. (2025) · ACM International Conference on Interaction Design and Children (IDC) · doi:10.1145/3713043.3731534
E-Books for High School Computer Science Teachers
Barbara, J. E., & Mark, G. (2022) · Emerald Group Publishing Limited eBooks · doi:10.1108/978-1-64802-908-020251011
Designing pedagogically effective activities for learning programming in a mobile tutor.
Chee, G. V. F. (2019) · UC Research Repository (University of Canterbury) · doi:10.26021/1887
Learning to Program != "One-Size-Fits-All": Exploring Variations of Parsons Problems as ScaffoldingPreprint
Haynes-Magyar, C. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2512.22407
Investigating the potential of using worked examples to help resolve issues in a GitHub project
Rocha, J. V. S., Wiese, I., Polato, I., Silva, M. A. G., Ré, R., Steinmacher, I., & Nakamura, W. T. (2026) · Journal of Systems and Software · doi:10.1016/j.jss.2026.112810
Maragkou, V., Metafas, D., & Rangoussi, M. (2026) · IEEE Global Engineering Education Conference (EDUCON) · doi:10.1109/educon67543.2026.11574093
Accessible Keyboard Controls for Block Ordering Problems
Fowles, P., Stefik, A., Blaser, B., Smith, D. H., & Poulsen, S. (2026) · ACM Conference on Innovation and Technology in Computer Science Education (ITiCSE) · doi:10.1145/3803400.3809324
CodeOwl: Automatic Generation of Tiered Parsons Problems for Introductory ProgrammingPreprint
Cisternino, L., Obermüller, F., & Fraser, G. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2607.12460
Personalized Assessments from Personal Artifacts
Zhang, Y., Savelka, J., Goldstein, S. C., & Sakr, M. F. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37982-5_56
Scaffolding Autocomplete: Improving Guidance for Learners using Generative Code Suggestions
Prather, J. E., MacNeil, S., Luxton-Reilly, A., Margulieux, L. E., Reeves, B. N., Denny, P. C., Leinonen, J., Homer, J., Nabid, R. A., & Rossetti, R. L. (2026) · ACM Conference on International Computing Education Research (ICER) · doi:10.1145/3765964.3811654
Self-citationGroße-Bölting, G., & Weißenrieder, F. (2026) · ACM Transactions on Computing Education · doi:10.1145/3844735
Isenegger, K. (2025) · Illinois Digital Environment for Access to Learning and Scholarship (University of Illinois at Urbana-Champaign)
Discovering, autogenerating, and evaluating distractors for Parsons problems in CS1
Smith, D. H., IV (2025) · Illinois Digital Environment for Access to Learning and Scholarship (University of Illinois at Urbana-Champaign)
Metadata
- Authors
- Ericson, B. J., Margulieux, L. E., & Rick, J.
- Affiliations
- Georgia Institute of Technology (US)
- Georgia State University (US)
- Venue
- Proceedings of the 17th Koli Calling International Conference on Computing Education Research
- Publisher
- ACM
- Published
- 2017-11-16
- Pages
- 20-29
- OpenAlex
- W2766230222
- Semantic Scholar
- 2f1b7c75a3cc2f33fc34575bbe77eeb1f336debc
- Cited by
- 121
- References
- 56
- Citations by year
- 2026: 6 · 2025: 9 · 2024: 13 · 2023: 21 · 2022: 18 · 2021: 19 · 2020: 18 · 2019: 14 · 2018: 3
- FWCI
- 8.82
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- Koli Calling International Conference on Computing Education Research (publishedVersion)
- Funders
- National Science Foundation (1432300 and 1138378)
- Topics
- Physical Sciences › Computer Science › Computer Science Applications › Teaching and Learning Programming
- Social Sciences › Education › Educational Robotics and Engineering
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Visual and Cognitive Learning Processes
- Keywords
- Parsons problems, distractors, code completion, writing software, programming education, cognitive load, learning efficiency, knowledge retention, summative assessment
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.