The DOI matches its Crossref record; 1 of the 1 references it cites were found in a registry or an index.
Evaluating a new exam question: Parsons problems
Abstract
Common exam practice centres around two question types: code tracing (reading) and code writing. It is commonly believed that code tracing is easier than code writing, but it seems obvious that different skills are needed for each. These problems also differ in their value on an exam. Pedagogically, code tracing on paper is an authentic task whereas code writing on paper is less so. Yet, few instructors are willing to forgo the code writing question on an exam. Is there another way, easier to grade, that captures the "problem solving through code creation process" we wish to examine? In this paper we propose Parson's puzzle-style problems for this purpose. We explore their potential both qualitatively, through interviews, and quantitatively through a set of CS1 exams. We find notable correlation between Parsons scores and code writing scores. We find low correlation between code writing and tracing and between Parsons and tracing. We also make the case that marks from a Parsons problem make clear what students don't know (specifically, in both syntax and logic) much less ambiguously than marks from a code writing problem. We make recommendations on the design of Parsons problems for the exam setting, discuss their potential uses and urge further investigations of Parsons problems for assessment of CS1 students.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A multi-national study of reading and tracing skills in novice programmers
Lister, R., Adams, E. S., Fitzgerald, S., Fone, W., Hamer, J., Lindholm, M., McCartney, R., Moström, J. E., Sanders, K., Seppälä, O., et al. (2004) · ACM SIGCSE Bulletin · doi:10.1145/1041624.1041673 · Cited by 575
Self-citation
Cited by
Introductory programming: a systematic literature review
Luxton-Reilly, A., Simon, Albluwi, I., Becker, B. A., Giannakos, M., Kumar, A. N., Ott, L., Paterson, J., Scott, M. J., Sheard, J., et al. (2018) · Proceedings Companion of the 23rd Annual ACM Conference on Innovation and Technology in Computer Science Education · doi:10.1145/3293881.3295779 · Cited by 485
Self-citationDesigning for deeper learning in a blended computer science course for middle school students
Grover, S., Pea, R., & Cooper, S. (2015) · Computer Science Education · doi:10.1080/08993408.2015.1033142 · Cited by 410
Solving parsons problems versus fixing and writing code
Ericson, B. J., Margulieux, L. E., & Rick, J. (2017) · Proceedings of the 17th Koli Calling International Conference on Computing Education Research · doi:10.1145/3141880.3141895 · Cited by 121
Teach AI How to Code: Using Large Language Models as Teachable Agents for Programming Education
Jin, H., Lee, S., Shin, H., & Kim, J. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642349 · Cited by 99
CS1-LLM: Integrating LLMs into CS1 Instruction
Vadaparty, A., Zingaro, D., Smith IV, D. H., Padala, M., Alvarado, C., Gorson Benario, J., & Porter, L. (2024) · Proceedings of the 2024 on Innovation and Technology in Computer Science Education V. 1 · doi:10.1145/3649217.3653584 · Cited by 77
Metadata
- Authors
- Denny, P., Luxton-Reilly, A., & Simon, B.
- Venue
- Proceedings of the Fourth international Workshop on Computing Education Research
- Publisher
- ACM
- Published
- 2008-09-06
- Pages
- 113-124
- OpenAlex
- W1987790173
- Cited by
- 203
- References
- 1
- FWCI
- 6.73
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.