The DOI matches its Crossref record; 4 of the 4 references it cites were found in a registry or an index.
What Types of Defects Are Really Discovered in Code Reviews?
Abstract
Research on code reviews has often focused on defect counts instead of defect types, which offers an imperfect view of code review benefits. In this paper, we classified the defects of nine industrial (C/C++) and 23 student (Java) code reviews, detecting 388 and 371 defects, respectively. First, we discovered that 75 percent of defects found during the review do not affect the visible functionality of the software. Instead, these defects improved software evolvability by making it easier to understand and modify. Second, we created a defect classification consisting of functional and evolvability defects. The evolvability defect classification is based on the defect types found in this study, but, for the functional defects, we studied and compared existing functional defect classifications. The classification can be useful for assigning code review roles, creating checklists, assessing software evolvability, and building software engineering tools. We conclude that, in addition to functional defects, code reviews find many evolvability defects and, thus, offer additional benefits over execution-based quality assurance methods that cannot detect evolvability defects. We suggest that code reviews may be most valuable for software products with long life cycles as the value of discovering evolvability defects in them is greater than for short life cycle systems.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Orthogonal defect classification-a concept for in-process measurements
Chillarege, R., Bhandari, I. S., Chaar, J. K., Halliday, M. J., Moebus, D. S., Ray, B. K., & Wong, M.-Y. (1992) · IEEE Transactions on Software Engineering · doi:10.1109/32.177364 · Cited by 792
The WyCash portfolio management system
Cunningham, W. (1992) · Addendum to the proceedings on Object-oriented programming systems, languages, and applications (Addendum) - OOPSLA '92 · doi:10.1145/157709.157715 · Cited by 742
How are Java software developers using the Eclipse IDE?
Murphy, G. C., Kersten, M., & Findlater, L. (2006) · IEEE Software · doi:10.1109/ms.2006.105 · Cited by 413
Treating Ordinal Scales as Interval Scales: An Attempt To Resolve the Controversy
KNAPP, T. R. (1990) · Nursing Research · doi:10.1097/00006199-199003000-00019 · Cited by 271 · cited 1×
“The authors are aware of the controversy relating to the use of mean values from ordinal scale data [35], [41].”
Background
Cited by
An exploration of technical debt
Tom, E., Aurum, A., & Vidgen, R. (2013) · Journal of Systems and Software · doi:10.1016/j.jss.2012.12.052 · Cited by 438
An empirical study of the impact of modern code review practices on software quality
McIntosh, S., Kamei, Y., Adams, B., & Hassan, A. E. (2016) · Empirical Software Engineering · doi:10.1007/s10664-015-9381-9 · Cited by 332
(2014) · doi:10.1145/2597073.2597082 · Cited by 252
“Only two studies analysed the outcome of MCR: (1) Bacchelli and Bird investigated motivations and outcomes of MCR at Microsoft [5], and (2) Mäntylä and Lassenius classified the types of defects found in review on university and three industrial software systems [32].”
BackgroundMethodResultInfluential(2018) · doi:10.1145/3274404 · Cited by 122
Code Review Automation: Strengths and Weaknesses of the State of the Art
Tufano, R., Dabić, O., Mastropaolo, A., Ciniselli, M., & Bavota, G. (2024) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2023.3348172 · Cited by 48
Metadata
- Authors
- Mantyla, M. V., & Lassenius, C.
- Venue
- IEEE Transactions on Software Engineering
- Published
- 2009-05
- Volume
- 35
- Issue
- 3
- Pages
- 430-448
- ISSN
- 0098-5589 (ISSN-L)
- OpenAlex
- W2142245496
- Cited by
- 204
- References
- 4
- FWCI
- 14.51
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: What Types of Defects Are Really Discovered in Code Reviews? https://app.cochat.ai/papers/10.1109/tse.2008.71
- The paper
- Mantyla, M. V., & Lassenius, C. (2009). What Types of Defects Are Really Discovered in Code Reviews? IEEE Transactions on Software Engineering. https://doi.org/10.1109/tse.2008.71
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.