Static Analyses in Python Programming Courses
Abstract
Students learning to program often rely on feedback from the compiler and from instructor-provided test cases to help them identify errors in their code. This feedback focuses on functional correctness, and the output, which is often phrased in technical language, may be difficult to for novices to understand or effectively use. Static analyses may be effective as a complementary aid, as they can highlight common errors that may be potential sources of problems. In this paper, we introduce PyTA, a wrapper for pylint that provides custom checks for common novice errors as well as improved messages to help students fix the errors that are found. We report on our experience integrating PyTA into an existing online system used to deliver programming exercises to CS1 students and evaluate it by comparing exercise submissions collected from the integrated system to previously collected data. This analysis demonstrates that, for students who chose to read the PyTA output, we observed a decrease in time to solve errors, occurrences of repeated errors, and submissions to complete a programming problem. This suggests that PyTA, and static analyses in general, may help students identify functional issues in their code not highlighted by compiler feedback and that static analysis output may help students more quickly identify debug their code.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Novices often struggle with indentation.
“As novices often struggle with indentation [24,27]”
From Using Large Language Models to Enhance Programming Error Messages (2023) · cites this paper as [27]
Cites
Using Static Analysis to Find Bugs
Ayewah, N., Pugh, W., Hovemeyer, D., Morgenthaler, J. D., & Penix, J. (2008) · IEEE Software · doi:10.1109/ms.2008.130 · Cited by 536 · see citing works
Enhancing syntax error messages appears ineffectual
Denny, P. C., Luxton-Reilly, A., & Carpenter, D. (2014) · doi:10.1145/2591708.2591748 · Cited by 148 · see citing works
On Compiler Error Messages: What TheySayand What TheyMean
Traver, V. J. (2010) · Advances in Human-Computer Interaction · doi:10.1155/2010/602570 · Cited by 128 · see citing works
An Effective Approach to Enhancing Compiler Error Messages
Becker, B. A. (2016) · ACM Technical Symposium on Computer Science Education (SIGCSE) · doi:10.1145/2839509.2844584 · Cited by 125 · see citing works
Identifying and correcting Java programming errors for introductory computer science students
Hristova, M., Misra, A., Rutter, M., & Mercuri, R. T. (2003) · ACM SIGCSE Bulletin · doi:10.1145/792548.611956 · Cited by 120 · see citing works
Murphy, L., Lewandowski, G., McCauley, R. A., Simon, B., Thomas, L., & Zander, C. (2008) · ACM SIGCSE Bulletin · doi:10.1145/1352322.1352191 · Cited by 118 · see citing works
Investigating Static Analysis Errors in Student Java Programs
Edwards, S. H., Kandru, N., & Rajagopal, M. B. M. M. (2017) · ACM Conference on International Computing Education Research (ICER) · doi:10.1145/3105726.3106182 · Cited by 87 · see citing works
An analysis of patterns of debugging among novice computer science students
Ahmadzadeh, M., Elliman, D., & Higgins, C. (2005) · ACM SIGCSE Bulletin · doi:10.1145/1151954.1067472 · Cited by 85 · see citing works
Do Enhanced Compiler Error Messages Help Students?: Results Inconclusive.
Pettit, R. S., Homer, J., & Gee, R. (2017) · Proceedings of the 2017 ACM SIGCSE Technical Symposium on Computer Science Education · doi:10.1145/3017680.3017768 · Cited by 66 · cited 2× · see citing works
“performed similar work at their institution and concur, stating that despite positive feedback from both students and instructors, they saw no measurable benefit in terms of a reduction in student submissions to programming problems [13].”
BackgroundThe Effects of Enhanced Compiler Error Messages on a Syntax Error Debugging Test
Becker, B. A., Goslin, K., & Glanville, G. (2018) · ACM Technical Symposium on Computer Science Education (SIGCSE) · doi:10.1145/3159450.3159461 · Cited by 62 · see citing works
Empowering students and building confidence in novice programmers through gauntlet
Flowers, T. R., Carver, C. A., & Jackson, J. (2005) · 2022 IEEE Frontiers in Education Conference (FIE) · doi:10.1109/fie.2004.1408551 · Cited by 56 · see citing works
Facilitating code-writing in PI classes
Zingaro, D., Cherenkova, Y., Karpova, O., & Petersen, A. (2013) · ACM Technical Symposium on Computer Science Education (SIGCSE) · doi:10.1145/2445196.2445369 · Cited by 44 · see citing works
Self-citationNienaltowski, M.-H., Pedroni, M., & Meyer, B. (2008) · ACM SIGCSE Bulletin · doi:10.1145/1352322.1352192 · Cited by 27 · see citing works
Study of novice programmers using Eclipse and Gild
Rigby, P. C., & Thompson, S. (2005) · doi:10.1145/1117696.1117718 · Cited by 24 · see citing works
Schorsch, T. (1995) · ACM SIGCSE Bulletin · doi:10.1145/199691.199769 · Cited by 24 · see citing works
Static analysis of source code written by novice programmers
Delev, T., & Gjorgjevikj, D. (2017) · IEEE Global Engineering Education Conference (EDUCON) · doi:10.1109/educon.2017.7942942 · Cited by 23 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Automated Grading and Feedback Tools for Programming Education: A Systematic Review
Messer, M., Brown, N. C. C., Kölling, M., & Shi, M. (2024) · ACM Transactions on Computing Education · doi:10.1145/3636515 · Cited by 262 · see citing works
Students’ Perceptions and Preferences of Generative Artificial Intelligence Feedback for Programming
Zhang, Z., Dong, Z., Shi, Y., Price, T., Matsuda, N., & Xu, D. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i21.30372 · Cited by 54 · see citing works
- Authors
- Liu, D., & Petersen, A.
- Affiliations
- University of Toronto (CA)
- ORCID
- Petersen, Andrew
- Venue
- Proceedings of the 50th ACM Technical Symposium on Computer Science Education
- Publisher
- ACM
- Published
- 2019-02-22
- Pages
- 666-671
- OpenAlex
- W2916351062
- Cited by
- 46
- References
- 16
- Citations by year
- 2026: 3 · 2025: 12 · 2024: 9 · 2023: 10 · 2022: 5 · 2021: 1 · 2020: 3 · 2019: 3
- FWCI
- 3.47
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM Technical Symposium on Computer Science Education (SIGCSE) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Science Applications › Teaching and Learning Programming
- Physical Sciences › Computer Science › Artificial Intelligence › Logic, programming, and type systems
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Keywords
- static analysis, Pylint, novice programmers, programming education, CS1, error feedback, debugging, programming exercises, error messages
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
