How should compilers explain problems to developers?
The DOI matches its Crossref record; 6 of the 6 references it cites were found in a registry or an index.
Abstract
Compilers primarily give feedback about problems to developers through the use of error messages. Unfortunately, developers routinely find these messages to be confusing and unhelpful. In this paper, we postulate that because error messages present poor explanations, theories of explanation---such as Toulmin's model of argument---can be applied to improve their quality. To understand how compilers should present explanations to developers, we conducted a comparative evaluation with 68 professional software developers and an empirical study of compiler error messages found in Stack Overflow questions across seven different programming languages.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
STATISTICAL METHODS FOR ASSESSING AGREEMENT BETWEEN TWO METHODS OF CLINICAL MEASUREMENT
Martin Bland, J., & Altman, D. (1986) · The Lancet · doi:10.1016/s0140-6736(86)90837-8 · Cited by 47,965
Generalization in quantitative and qualitative research: Myths and strategies
Polit, D. F., & Beck, C. T. (2010) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2010.06.004 · Cited by 1,767
Memoing in qualitative research: Probing data and processes
Birks, M., Chapman, Y., & Francis, K. (2008) · Journal of Research in Nursing · doi:10.1177/1744987107081254 · Cited by 1,519
Using Numbers in Qualitative Research
Maxwell, J. A. (2010) · Qualitative Inquiry · doi:10.1177/1077800410364740 · Cited by 985 · cited 1×
“To characterize what types of argument structures are found in accepted answers, we used quasi-statistics [27]—essentially, a process of transforming qualitative data to simple counts—to aid in the interpretation of the Stack Overflow data.”
MethodHardaker, C. (2010) · Journal of Politeness Research. Language, Behaviour, Culture · doi:10.1515/jplr.2010.011 · Cited by 597
Do Developers Read Compiler Error Messages?
Barik, T., Smith, J., Lubick, K., Holmes, E., Feng, J., Murphy-Hill, E., & Parnin, C. (2017) · 2017 IEEE/ACM 39th International Conference on Software Engineering (ICSE) · doi:10.1109/icse.2017.59 · Cited by 86 · cited 1×
“[2] conducted an eye-tracking study with developers and found that they spent up to 25% of their task time on reading error messages.”
BackgroundSelf-citation
Cited by
Becker, B. A., Denny, P., Pettit, R., Bouchard, D., Bouvier, D. J., Harrington, B., Kamil, A., Karkare, A., McDonald, C., Osera, P.-M., et al. (2019) · Proceedings of the Working Group Reports on Innovation and Technology in Computer Science Education · doi:10.1145/3344429.3372508 · Cited by 199
On Designing Programming Error Messages for Novices: Readability and its Constituent Factors
Denny, P., Prather, J., Becker, B. A., Mooney, C., Homer, J., Albrecht, Z. C., & Powell, G. B. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445696 · Cited by 60
“While experienced developers may fnd poor error messages an irritation, for novices they can be paralyzing [5].”
BackgroundError Message Readability and Novice Debugging Performance
Denny, P., Prather, J., & Becker, B. A. (2020) · Proceedings of the 2020 ACM Conference on Innovation and Technology in Computer Science Education · doi:10.1145/3341525.3387384 · Cited by 50
Danilova, A., Naiakshina, A., & Smith, M. (2020) · Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering · doi:10.1145/3377811.3380387 · Cited by 25
“[11] investigated compiler errors and how developers understood error messages.”
How statically-typed functional programmers write code
Lubin, J., & Chasins, S. E. (2021) · Proceedings of the ACM on Programming Languages · doi:10.1145/3485532 · Cited by 18
“Thus, while prior work has investigated how compilers should communicate errors to programmers (Barik et al., 2018, Becker et al., 2018, Nienaltowski et al., 2008), reimagining compiler feedback as sometimes being directive in nature rather than strictly corrective could align it more closely to…”
Background
Metadata
- Authors
- Barik, T., Ford, D., Murphy-Hill, E., & Parnin, C.
- Venue
- Proceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering
- Publisher
- ACM
- Published
- 2018-10-26
- Pages
- 633-643
- OpenAlex
- W2899324569
- Cited by
- 41
- References
- 6
- FWCI
- 3.39
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: How should compilers explain problems to developers? https://app.cochat.ai/papers/10.1145/3236024.3236040
- The paper
- Barik, T., Ford, D., Murphy-Hill, E., & Parnin, C. (2018). How should compilers explain problems to developers? Proceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering. https://doi.org/10.1145/3236024.3236040
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.