The DOI matches its Crossref record; 84 of the 84 references it cites were found in a registry or an index.
An Empirical Investigation into Programming Language Syntax
Abstract
Recent studies in the literature have shown that syntax remains a significant barrier to novice computer science students in the field. While this syntax barrier is known to exist, whether and how it varies across programming languages has not been carefully investigated. For this article, we conducted four empirical studies on programming language syntax as part of a larger analysis into the, so called, programming language wars. We first present two surveys conducted with students on the intuitiveness of syntax, which we used to garner formative clues on what words and symbols might be easy for novices to understand. We followed up with two studies on the accuracy rates of novices using a total of six programming languages: Ruby, Java, Perl, Python, Randomo, and Quorum. Randomo was designed by randomly choosing some keywords from the ASCII table (a metaphorical placebo). To our surprise, we found that languages using a more traditional C-style syntax (both Perl and Java) did not afford accuracy rates significantly higher than a language with randomly generated keywords, but that languages which deviate (Quorum, Python, and Ruby) did. These results, including the specifics of syntax that are particularly problematic for novices, may help teachers of introductory programming courses in choosing appropriate first languages and in helping students to overcome the challenges they face with syntax.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Instructors report that they spend a considerable amount of time helping novices with these often cryptic messages [15,31,32,37].”
From Using Large Language Models to Enhance Programming Error Messages (2023) · cites this paper as [37]
Citations
Cites
Kiczales, G. (1996) · ACM Computing Surveys · doi:10.1145/242224.242420 · Cited by 5,146
An Easy Guide to Factor Analysis
Kline, P. (2014) · Routledge eBooks · doi:10.4324/9781315788135 · Cited by 3,380
Resnick, M., Maloney, J., Monroy-Hernández, A., Rusk, N., Eastmond, E., Brennan, K., Millner, A., Rosenbaum, E., Silver, J., Silverman, B., et al. (2009) · Communications of the ACM · doi:10.1145/1592761.1592779 · Cited by 3,333
The Scratch Programming Language and Environment
Maloney, J. H., Resnick, M., Rusk, N., Silverman, B., & Eastmond, E. (2010) · ACM Transactions on Computing Education · doi:10.1145/1868358.1868363 · Cited by 1,423
Usability Analysis of Visual Programming Environments: A ‘Cognitive Dimensions’ Framework
Green, T. R. G., & Petre, M. (1996) · Journal of Visual Languages & Computing · doi:10.1006/jvlc.1996.0009 · Cited by 1,252
Pinker, S. A. (1991) · Science · doi:10.1126/science.1857983 · Cited by 1,061
Lowering the barriers to programming
Kelleher, C. L., & Pausch, R. F. (2005) · ACM Computing Surveys · doi:10.1145/1089733.1089734 · Cited by 887
The Cornell program synthesizer
Teitelbaum, T., & Reps, T. (1981) · Communications of the ACM · doi:10.1145/358746.358755 · Cited by 712
The state of the art in end-user software engineering
Ko, A. J., Abraham, R., Beckwith, L., Blackwell, A., Burnett, M., Erwig, M., Scaffidi, C., Lawrance, J., Lieberman, H., Myers, B., et al. (2011) · ACM Computing Surveys · doi:10.1145/1922649.1922658 · Cited by 633
A multi-national study of reading and tracing skills in novice programmers
Lister, R., Adams, E. S., Fitzgerald, S., Fone, W., Hamer, J., Lindholm, M., McCartney, R., Moström, J. E., Sanders, K., Seppälä, O., et al. (2004) · ACM SIGCSE Bulletin · doi:10.1145/1041624.1041673 · Cited by 575
Flagged first, then by citations
Research works printed but not linked: 9
- Binkley D., Proceedings of the IEEE 17th International Conference on Program Comprehension (ICPC’09)
- 10.5555/646147.679038 · DOI
- Brown M. H., Proceedings of the IEEE Workshop on Visual Languages (VL’91)
- Delorey D. P., 2009, Proceedings of the Psychology of Programming Interest Group Conference (PPIG
- Feigenspan J., Proceedings of the IEEE 20th International Conference on Program Comprehension (ICPC’12)
- Kleinschmager S., Proceedings of the IEEE 20th International Conference on Program Comprehension (ICPC’12)
- Pennington N., 1987, Empirical Studies of Programmers: Second Workshop, G. M. Olson, S. Sheppard, E. Soloway, and B. Shneiderman Eds., Greenwood Publishing Group Inc., 100--113
- Stefik A., Proceedings of the 17th IEEE International Conference on Program Comprehension (ICPC’09)
- Vogt W. P., Quantitative Research Methods for Professionals in Education and Other Fields
Cited by
Sorted by citations
Introductory programming: a systematic literature review
Luxton-Reilly, A., Simon, Al-Bluwi, I., Becker, B. A., Giannakos, M. N., Kumar, A. N., Ott, L. M., Paterson, J. H., Scott, M., Sheard, J., et al. (2018) · doi:10.1145/3293881.3295779 · Cited by 511
Bau, D., Gray, J., Kelleher, C. L., Sheldon, J., & Turbak, F. A. (2017) · Communications of the ACM · doi:10.1145/3015455 · Cited by 326
Altadmri, A., & Brown, N. C. C. (2015) · ACM Technical Symposium on Computer Science Education (SIGCSE) · doi:10.1145/2676723.2677258 · Cited by 261
Learner-Centered Design of Computing Education: Research on Computing for Everyone
Guzdial, M. (2015) · Synthesis lectures on human-centered informatics · doi:10.2200/s00684ed1v01y201511hci033 · Cited by 260
Using Large Language Models to Enhance Programming Error Messages
Leinonen, J., Hellas, A., Sarsa, S., Reeves, B., Denny, P., Prather, J., & Becker, B. A. (2023) · Proceedings of the 54th ACM Technical Symposium on Computer Science Education V. 1 · doi:10.1145/3545945.3569770 · Cited by 235
Metadata
- Authors
- Stefik, A., & Siebert, S.
- Affiliations
- University of Nevada, Las Vegas (US)
- Washington University in St. Louis (US)
- ORCID
- Stefik, Andreas
- Venue
- ACM Transactions on Computing Education
- Published
- 2013-11
- Volume
- 13
- Issue
- 4
- Pages
- 1-40
- ISSN
- 1946-6226 (ISSN-L)
- OpenAlex
- W2157148654
- Cited by
- 295
- References
- 84
- Citations by year
- 2026: 14 · 2025: 13 · 2024: 21 · 2023: 19 · 2022: 19 · 2021: 19 · 2020: 34 · 2019: 60 · 2018: 23 · 2017: 22 · 2016: 17 · 2015: 16 · 2014: 15 · 2013: 2
- FWCI
- 25.18
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- ACM Transactions on Computing Education (publishedVersion)
- Funders
- Division of Computer and Network Systems (CNS-0940521)
- Topics
- Physical Sciences › Computer Science › Computer Science Applications › Teaching and Learning Programming
- Physical Sciences › Computer Science › Software › Logic, programming, and type systems
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Keywords
- programming language syntax, novice programmers, quorum, Python, ruby, Java, Perl, introductory programming
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
