Fix the First, Ignore the Rest: Dealing with Multiple Compiler Error Messages
The DOI matches its Crossref record; 30 of the 30 references it cites were found in a registry or an index.
Abstract
In order to help students learning to develop computer programs, several computing education researchers have analyzed the compiler error messages generated by novices' attempts to compile their programs. The goal is to help students diagnose the errors they make through the messages generated by the compiler. This paper builds on that previous work by applying a technique based on a heuristic well-known to programmers - fix the first error and ignore the rest - to the analysis of over 21 million compiler error messages from the Blackbox dataset. We find that the ranks and frequencies obtained by considering all error messages are generally consistent with previously published lists, but when we consider first messages only, these ranks and frequencies are different. These differences could have important implications for teaching, and can inform tool design and future research efforts.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
The BlueJ System and its Pedagogy
Kölling, M., Quig, B., Patterson, A., & Rosenberg, J. (2003) · Computer Science Education · doi:10.1076/csed.13.4.249.17496 · Cited by 484
37 Million Compilations: Investigating Novice Programming Mistakes in Large-Scale Student Data
Altadmri, A., & Brown, N. C. C. (2015) · Proceedings of the 46th ACM Technical Symposium on Computer Science Education · doi:10.1145/2676723.2677258 · Cited by 261
Identifying and correcting Java programming errors for introductory computer science students
Hristova, M., Misra, A., Rutter, M., & Mercuri, R. (2003) · Proceedings of the 34th SIGCSE technical symposium on Computer science education · doi:10.1145/611892.611956 · Cited by 201 · cited 2×
“Several authors have analyzed student understanding of error messages, developed tools that aempt to provide beer compiler error messages, and/or supplemented standard messages with additional information [7, 13, 16, 18, 19, 24, 28, 32].”
BackgroundAn analysis of patterns of debugging among novice computer science students
Ahmadzadeh, M., Elliman, D., & Higgins, C. (2005) · Proceedings of the 10th annual SIGCSE conference on Innovation and technology in computer science education · doi:10.1145/1067445.1067472 · Cited by 185
Blackbox: a large scale repository of novice programmers' activity
Brown, N. C. C., Kölling, M., McCall, D., & Utting, I. (2014) · Proceedings of the 45th ACM technical symposium on Computer science education · doi:10.1145/2538862.2538924 · Cited by 179 · cited 3×
“e list of most-frequent rst messages is notably dierent from the corresponding list for all messages, and from similar lists that have been reported before [11, 16, 20].”
BackgroundMethodEnhancing syntax error messages appears ineffectual
Denny, P., Luxton-Reilly, A., & Carpenter, D. (2014) · Proceedings of the 2014 conference on Innovation & technology in computer science education - ITiCSE '14 · doi:10.1145/2591708.2591748 · Cited by 148 · cited 2×
“Several authors have analyzed student understanding of error messages, developed tools that aempt to provide beer compiler error messages, and/or supplemented standard messages with additional information [7, 13, 16, 18, 19, 24, 28, 32].”
BackgroundAll syntax errors are not equal
Denny, P., Luxton-Reilly, A., & Tempero, E. (2012) · Proceedings of the 17th ACM annual conference on Innovation and technology in computer science education · doi:10.1145/2325296.2325318 · Cited by 144 · cited 2×
“[14] collected errors from student exercises using a custom “drill and practice” tool.”
BackgroundMethodAn Effective Approach to Enhancing Compiler Error Messages
Becker, B. A. (2016) · ACM Technical Symposium on Computer Science Education (SIGCSE) · doi:10.1145/2839509.2844584 · Cited by 125 · cited 1×
“[4, 5, 7] used an editor designed to collect and enhance error messages generated by the standard JDK compiler.”
MethodSelf-citationNovice Java Programming Mistakes: Large-Scale Data vs. Educator Beliefs
Brown, N. C. C., & Altadmri, A. (2017) · ACM Transactions on Computing Education · doi:10.1145/2994154 · Cited by 107 · cited 4×
“As an example of how such lists can have long lifetimes in informing other research, in 2017 Brown and Altadmri [10] used a set of 18 errors derived from the 20 originally presented in 2003 by Hristova et al.”
BackgroundMethodInfluentialEffective compiler error message enhancement for novice programming students
Becker, B. A., Glanville, G., Iwashima, R., McDonnell, C., Goslin, K., & Mooney, C. (2016) · Computer Science Education · doi:10.1080/08993408.2016.1225464 · Cited by 101 · cited 3×
“[4, 5, 7] used an editor designed to collect and enhance error messages generated by the standard JDK compiler.”
BackgroundMethodSelf-citation
Research works printed but not linked: 2
- Murray Cormac, A Comparative Study of Java Compiler Error Profiles Using the Blackbox Dataset. Master's thesis
- Suzanne Marie Thompson. 2004. An Exploratory Study of Novice Programming Experiences and Errors. Master's thesis. Univ. of Victoria. Suzanne Marie Thompson. 2004. An Exploratory Study of Novice Programming Experiences and Errors. Master's thesis. Univ. of Victoria.
Cited by
Prather, J., Reeves, B. N., Denny, P., Becker, B. A., Leinonen, J., Luxton-Reilly, A., Powell, G., Finnie-Ansley, J., & Santos, E. A. (2024) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3617367 · Cited by 256
Self-citationBecker, B. A., Denny, P., Pettit, R., Bouchard, D., Bouvier, D. J., Harrington, B., Kamil, A., Karkare, A., McDonald, C., Osera, P.-M., et al. (2019) · Proceedings of the Working Group Reports on Innovation and Technology in Computer Science Education · doi:10.1145/3344429.3372508 · Cited by 199
“…(Ben-Ari, 2007) [24], undecipherable (Traver, 2010) [202], intimidating (Hartz, 2012) [84], still very obviously less helpful than they could be (McCall & Kölling, 2015) [139], inscrutable (Ko, 2017) 1 , frustrating [again, 13 years after [72]] and a barrier to progress (Becker et al. , 2018) [21].”
BackgroundSelf-citation(2019) · doi:10.1145/3287324.3287374 · Cited by 104
Self-citationWhat Do CS1 Syllabi Reveal About Our Expectations of Introductory Programming Students?
Becker, B. A., & Fitzpatrick, T. (2019) · Proceedings of the 50th ACM Technical Symposium on Computer Science Education · doi:10.1145/3287324.3287485 · Cited by 63
Self-citationThe Effects of Enhanced Compiler Error Messages on a Syntax Error Debugging Test
Becker, B. A., Goslin, K., & Glanville, G. (2018) · Proceedings of the 49th ACM Technical Symposium on Computer Science Education · doi:10.1145/3159450.3159461 · Cited by 61
Self-citation
Metadata
- Authors
- Becker, B. A., Murray, C., Tao, T., Song, C., McCartney, R., & Sanders, K.
- Affiliations
- University College Dublin (IE)
- Fudan University (CN)
- University of Connecticut (US)
- Rhode Island College (US)
- Venue
- Proceedings of the 49th ACM Technical Symposium on Computer Science Education
- Publisher
- ACM
- Published
- 2018-02-21
- Pages
- 634-639
- OpenAlex
- W2791265653
- Cited by
- 61
- References
- 30
- Citations by year
- 2025: 12 · 2024: 7 · 2023: 6 · 2022: 6 · 2021: 12 · 2020: 6 · 2019: 8 · 2018: 4
- FWCI
- 8.49
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM Technical Symposium on Computer Science Education (SIGCSE) (publishedVersion)
- Funders
- University of Kent
- Topics
- Physical Sciences › Computer Science › Computer Science Applications › Teaching and Learning Programming
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Computer Science › Software › Logic, programming, and type systems
- Keywords
- compiler error messages, novice programmers, programming education, error diagnosis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Fix the First, Ignore the Rest: Dealing with Multiple Compiler Error Messages. https://app.cochat.ai/papers/10.1145/3159450.3159453
- The paper
- Becker, B. A., Murray, C., Tao, T., Song, C., McCartney, R., & Sanders, K. (2018). Fix the First, Ignore the Rest: Dealing with Multiple Compiler Error Messages. Proceedings of the 49th ACM Technical Symposium on Computer Science Education. https://doi.org/10.1145/3159450.3159453
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.