The Compound Nature of Novice Programming Assessments
The DOI matches its Crossref record; 6 of the 6 references it cites were found in a registry or an index.
Abstract
Failure rates in introductory programming courses are notoriously high, and researchers have noted that students struggle with the assessments that we typically use to evaluate programming ability. Current assessment practices in introductory courses consist predominantly of questions that involve a multitude of different concepts and facts. Students with fragile knowledge in any of the areas required may be unable to produce a working solution, even when they may know most of the required material. These assessments also make it difficult for a teacher to distinguish between the specific information that a student does and does not know.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Learning and Teaching Programming: A Review and Discussion
Robins, A., Rountree, J., & Rountree, N. (2003) · Computer Science Education · doi:10.1076/csed.13.2.137.14200 · Cited by 1,736 · cited 2×
“3013500 learning to program is intrinsically difficult [3, 29].”
BackgroundFailure rates in introductory programming revisited
Watson, C., & Li, F. W. B. (2014) · Proceedings of the 2014 conference on Innovation & technology in computer science education - ITiCSE '14 · doi:10.1145/2591708.2591749 · Cited by 639 · cited 1×
“Yet the pass rates of many introductory programming courses remain an area of concern to the Computer Science Education community [4, 29, 43], and research on student performance has found that students in their first year of programming frequently struggle to both read and write relatively short programs [15, 23, 46].”
BackgroundA multi-national study of reading and tracing skills in novice programmers
Lister, R., Adams, E. S., Fitzgerald, S., Fone, W., Hamer, J., Lindholm, M., McCartney, R., Moström, J. E., Sanders, K., Seppälä, O., et al. (2004) · ACM SIGCSE Bulletin · doi:10.1145/1041624.1041673 · Cited by 575 · cited 4×
“The code shown in Figure 2 has been adapted from the Leeds study [15] by converting it from Java into Python.”
BackgroundMethodInfluentialRelationships between reading, tracing and writing skills in introductory programming
Lopez, M., Whalley, J., Robbins, P., & Lister, R. (2008) · Proceedings of the Fourth international Workshop on Computing Education Research · doi:10.1145/1404520.1404531 · Cited by 363
An Empirical Investigation into Programming Language Syntax
Stefik, A., & Siebert, S. (2013) · ACM Transactions on Computing Education · doi:10.1145/2534973 · Cited by 295 · cited 1×
“A number of recent studies have shown that syntax errors pose a significant barrier to novice programmers [1, 5, 8, 9, 22, 24, 39].”
BackgroundAll syntax errors are not equal
Denny, P., Luxton-Reilly, A., & Tempero, E. (2012) · Proceedings of the 17th ACM annual conference on Innovation and technology in computer science education · doi:10.1145/2325296.2325318 · Cited by 144 · cited 3×
“Additionally, novice programmers are observed to encounter significant problems understanding and using syntax correctly [1, 8, 9].”
BackgroundSelf-citation
Cited by
Introductory programming: a systematic literature review
Luxton-Reilly, A., Simon, Albluwi, I., Becker, B. A., Giannakos, M., Kumar, A. N., Ott, L., Paterson, J., Scott, M. J., Sheard, J., et al. (2018) · Proceedings Companion of the 23rd Annual ACM Conference on Innovation and Technology in Computer Science Education · doi:10.1145/3293881.3295779 · Cited by 485
Self-citationPlagiarism in Programming Assessments: A Systematic Review
Albluwi, I. (2020) · ACM Transactions on Computing Education · doi:10.1145/3371156 · Cited by 63
Towards an Analysis of Program Complexity From a Cognitive Perspective
Duran, R., Sorva, J., & Leite, S. (2018) · Proceedings of the 2018 ACM Conference on International Computing Education Research · doi:10.1145/3230977.3230986 · Cited by 44
“, like ourselves, analyzed individual programs in order to characterize their structure [50, 51]; their analysis and metrics are fundamentally different from ours in that they focused exclusively on syntactic features whereas we have focused on cognitive plans.”
BackgroundComputing Education Research Today
Fincher, S. A., Tenenberg, J., Dorn, B., Hundhausen, C., McCartney, R., & Murphy, L. (2019) · The Cambridge Handbook of Computing Education Research · doi:10.1017/9781108654555.003 · Cited by 7
Metadata
- Authors
- Luxton-Reilly, A., & Petersen, A.
- Venue
- Proceedings of the Nineteenth Australasian Computing Education Conference
- Publisher
- ACM
- Published
- 2017-01-31
- Pages
- 26-35
- OpenAlex
- W2582325117
- Cited by
- 66
- References
- 6
- FWCI
- 12.66
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.