The DOI matches its Crossref record; 57 of the 57 references it cites were found in a registry or an index.
Classifying Software Changes: Clean or Buggy?
Abstract
This paper introduces a new technique for finding latent software bugs called change classification. Change classification uses a machine learning classifier to determine whether a new software change is more similar to prior buggy changes, or clean changes. In this manner, change classification predicts the existence of bugs in software changes. The classifier is trained using features (in the machine learning sense) extracted from the revision history of a software project, as stored in its software configuration management repository. The trained classifier can classify changes as buggy or clean with 78% accuracy and 65% buggy change recall (on average). Change classification has several desirable qualities: (1) the prediction granularity is small (a change to a single file), (2) predictions do not require semantic information about the source code, (3) the technique works for a broad array of project types and programming languages, and (4) predictions can be made immediately upon completion of a change. Contributions of the paper include a description of the change classification approach, techniques for extracting features from source code and change histories, a characterization of the performance of change classification across 12 open source projects, and evaluation of the predictive power of different groups of features.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The Nature of Statistical Learning Theory
Vapnik, V. N. (1995) · doi:10.1007/978-1-4757-2440-0 · Cited by 39,138
Data Mining: Practical Machine Learning Tools and Techniques
(2011) · Elsevier eBooks · doi:10.1016/c2009-0-19715-5 · Cited by 25,437
UCI Repository of machine learning databases
Blake, C. L. (1998) · Medical Entomology and Zoology · Cited by 10,582
Witten, I. H. (2008) · Cited by 9,143
Text categorization with Support Vector Machines: Learning with many relevant features
Joachims, T. (1998) · Lecture Notes in Computer Science · doi:10.1007/bfb0026683 · Cited by 7,999
Machine learning in automated text categorization
Sebastiani, F. (2002) · ACM Computing Surveys · doi:10.1145/505282.505283 · Cited by 7,926
An introduction to latent semantic analysis
Landauer, T. K., Foltz, P. W., & Laham, D. (1998) · Discourse Processes · doi:10.1080/01638539809545028 · Cited by 4,880 · cited 3×
“For example, Maletic et al. [28], [29] extracted all features available in the source code via Latent Semantic Analysis (LSA) and then used this data to cluster software and to create relationships between the source code and other related software project documents.”
A Short Introduction to Boosting
Freund, Y., & Schapire, R. E. (1999) · Cited by 3,140
RCV1: A New Benchmark Collection for Text Categorization Research
Lewis, D., Yang, Y., Rose, T. G., & Li, F. (2004) · Goldsmiths (University of London) · doi:10.5555/1005332.1005345 · Cited by 2,503
UCI Repository of Machine Learning Databases
Murphy, P. M. (1994) · Medical Entomology and Zoology · Cited by 1,844
Introduction to Machine Learning
(2019) · Series in Machine Perception and Artificial Intelligence · doi:10.1142/9789811201967_0001 · Cited by 1,639
A brief introduction to boosting
Schapire, R. E. (1999) · International Joint Conference on Artificial Intelligence · Cited by 1,572
Extended static checking for Java
Flanagan, C., Leino, K. R. M., Lillibridge, M., Nelson, G., Saxe, J. B., & Stata, R. (2002) · Proceedings of the ACM SIGPLAN 2002 conference on Programming language design and implementation · doi:10.1145/512529.512558 · Cited by 1,340
Data Mining Static Code Attributes to Learn Defect Predictors
Menzies, T., Greenwald, J., & Frank, A. (2007) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2007.256941 · Cited by 1,215
Anvik, J., Hiew, L., & Murphy, G. C. (2006) · Proceedings of the 28th international conference on Software engineering · doi:10.1145/1134285.1134336 · Cited by 924
Empirical validation of object-oriented metrics on open source software for fault prediction
Gyimothy, T., Ferenc, R., & Siket, I. (2005) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2005.112 · Cited by 904
Mining version histories to guide software changes
Zimmermann, T., Zeller, A., Weissgerber, P., & Diehl, S. (2005) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2005.72 · Cited by 856
UCI Repository of Machine Learning Database
Newman, D. J. (1998) · Medical Entomology and Zoology · Cited by 824
Śliwerski, J., Zimmermann, T., & Zeller, A. (2005) · ACM SIGSOFT Software Engineering Notes · doi:10.1145/1082983.1083147 · Cited by 805
Use of relative code churn measures to predict system defect density
Nagappan, N., & Ball, T. (2005) · Proceedings of the 27th international conference on Software engineering - ICSE '05 · doi:10.1145/1062455.1062514 · Cited by 775 · cited 2×
“problem, predicting the bug density of each module by using its software change history [11], [36].”
BackgroundMethodPredicting fault incidence using software change history
Graves, T. L., Karr, A. F., Marron, J. S., & Siy, H. (2000) · IEEE Transactions on Software Engineering · doi:10.1109/32.859533 · Cited by 753
Predicting the location and number of faults in large software systems
Ostrand, T. J., Weyuker, E. J., & Bell, R. M. (2005) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2005.49 · Cited by 720
Feature selection for text categorization on imbalanced data
Zheng, Z., Wu, X., & Srihari, R. (2004) · ACM SIGKDD Explorations Newsletter · doi:10.1145/1007730.1007741 · Cited by 549
Recovering Documentation-to-Source-Code Traceability Links using Latent Semantic Indexing
Marcus, A., & Maletic, J. I. (2007) · Cited by 548
Populating a Release History Database from version control and bug tracking systems
Fischer, M., Pinzger, M., & Gall, H. (2004) · International Conference on Software Maintenance, 2003. ICSM 2003. Proceedings. · doi:10.1109/icsm.2003.1235403 · Cited by 525
Predicting Faults from Cached History
Kim, S., Zimmermann, T., Whitehead Jr., E. J., & Zeller, A. (2007) · 29th International Conference on Software Engineering (ICSE'07) · doi:10.1109/icse.2007.66 · Cited by 486
Self-citationIdentifying reasons for software changes using historic databases
Mockus, & Votta (2000) · Proceedings 2000 International Conference on Software Maintenance · doi:10.1109/icsm.2000.883028 · Cited by 472
Recovering documentation-to-source-code traceability links using latent semantic indexing
Marcus, A., & Maletic, J. I. (2003) · 25th International Conference on Software Engineering, 2003. Proceedings. · doi:10.1109/icse.2003.1201194 · Cited by 445
Predicting risk of software changes
Mockus, A., & Weiss, D. M. (2002) · Bell Labs Technical Journal · doi:10.1002/bltj.2229 · Cited by 429
Li, Z., & Zhou, Y. (2005) · Proceedings of the 10th European software engineering conference held jointly with 13th ACM SIGSOFT international symposium on Foundations of software engineering · doi:10.1145/1081706.1081755 · Cited by 428
Śliwerski, J., Zimmermann, T., & Zeller, A. (2005) · Proceedings of the 2005 international workshop on Mining software repositories - MSR '05 · doi:10.1145/1083142.1083147 · Cited by 395
DynaMine: finding common error patterns by mining software revision histories
Livshits, B., & Zimmermann, T. (2005) · Proceedings of the 10th European software engineering conference held jointly with 13th ACM SIGSOFT international symposium on Foundations of software engineering · doi:10.1145/1081706.1081754 · Cited by 330
Hipikat: recommending pertinent software development artifacts
Čubranić, D., & Murphy, G. C. (2003) · doi:10.5555/776816.776866 · Cited by 317
An evaluation of statistical spam filtering techniques
Zhang, L., Zhu, J., & Yao, T. (2004) · ACM Transactions on Asian Language Information Processing · doi:10.1145/1039621.1039625 · Cited by 315
Feature Engineering for Text Classification
Scott, S., & Matwin, S. (1999) · Cited by 307
Ostrand, T. J., Weyuker, E. J., & Bell, R. M. (2004) · Proceedings of the 2004 ACM SIGSOFT international symposium on Software testing and analysis · doi:10.1145/1007512.1007524 · Cited by 269
The distribution of faults in a large industrial software system
Ostrand, T. J., & Weyuker, E. J. (2002) · ACM SIGSOFT Software Engineering Notes · doi:10.1145/566171.566181 · Cited by 246
Preprocessing CVS data for fine-grained analysis
Zimmermann, T. (2004) · "International Workshop on Mining Software Repositories (MSR 2004)" W17S Workshop - 26th International Conference on Software Engineering · doi:10.1049/ic:20040466 · Cited by 214
The top ten list: dynamic fault prediction
Hassan, A. E., & Holt, R. C. (2005) · 21st IEEE International Conference on Software Maintenance (ICSM'05) · doi:10.1109/icsm.2005.91 · Cited by 207
Use of relative code churn measures to predict system defect density
Nagappan, N., & Ball, T. (2005) · Proceedings. 27th International Conference on Software Engineering, 2005. ICSE 2005. · doi:10.1109/icse.2005.1553571 · Cited by 201
Automatic mining of source code repositories to improve bug finding techniques
Williams, C. C., & Hollingsworth, J. K. (2005) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2005.63 · Cited by 198
Hipikat: recommending pertinent software development artifacts
Cubranic, D., & Murphy, G. C. (2003) · 25th International Conference on Software Engineering, 2003. Proceedings. · doi:10.1109/icse.2003.1201219 · Cited by 174
Finding latent code errors via machine learning over program executions
Brun, Y., & Ernst, M. D. (2004) · doi:10.5555/998675.999452 · Cited by 158
Kim, S., Pan, K., & Whitehead, E. J. (2006) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/1181775.1181781 · Cited by 152
Self-citationSHAW, M. C. (2001) · Elsevier eBooks · doi:10.1016/b978-081551447-3.50015-7 · Cited by 146
Extended static checking for Java
Cormac Flanagan, A., Leino, K. R. M., Lillibridge, M., Nelson, G., Saxe, J. B., & Stata, R. (2002) · ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI) · doi:10.1145/512557.512558 · Cited by 132
Facilitating software evolution research with kenyon
Bevan, J., Whitehead, E. J., Kim, S., et al. (2005) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/1081706.1081736 · Cited by 124
Self-citationAn approach to classify software maintenance requests
Lucca, G. A. D., Penta, M. D., & Gradara, S. (2003) · International Conference on Software Maintenance · doi:10.1109/icsm.2002.1167756 · Cited by 121
Finding latent code errors via machine learning over program executions
Brun, Y., & Ernst, M. D. (2004) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse.2004.1317470 · Cited by 81
An Introduction to Machine Learning
Hichem Sahli, B. (2020) · Wiley eBooks · doi:10.1002/9781119720492.ch7 · Cited by 68
The distribution of faults in a large industrial software system
Ostrand, T. J., & Weyuker, E. J. (2002) · ACM SIGSOFT Software Engineering Notes · doi:10.1145/566180.566181 · Cited by 61
DynaMine: finding common error patterns by mining software revision histories
Livshits, B., & Zimmermann, T. (2005) · Cited by 54
Traceability recovery by modeling programmer behavior
Antoniol, G., Casazza, C., & Cimitile, A. (2002) · Proceedings - Working Conference on Reverse Engineering · doi:10.1109/wcre.2000.891475 · Cited by 43
Training on errors experiment to detect fault-prone software modules by spam filter
Mizuno, O., & Kikuno, T. (2007) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/1287624.1287683 · Cited by 40
Traceability recovery in RAD software systems
Penta, M. D., Gradara, S., & Antoniol, G. (2003) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1109/wpc.2002.1021342 · Cited by 37
Predicting buggy changes inside an integrated development environment
Madhavan, J. T., & Whitehead, E. J. (2007) · Eclipse Technology eXchange · doi:10.1145/1328279.1328287 · Cited by 16
Spohrer, J., Soloway, E. M., & Pope, E. (1985) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/317456.317465 · Cited by 15
Cited by
The future of employment: How susceptible are jobs to computerisation?
Frey, C. B., & Osborne, M. A. (2017) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2016.08.019 · Cited by 8,727
A large-scale empirical study of just-in-time quality assurance
Kamei, Y., Shihab, E., Adams, B., Hassan, A. E., Mockus, A., Sinha, A., & Ubayashi, N. (2013) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2012.70 · Cited by 715
A systematic review of machine learning techniques for software fault prediction
Malhotra, R. (2015) · Applied Soft Computing · doi:10.1016/j.asoc.2014.11.023 · Cited by 620
Arisholm, E., Briand, L. C., & Johannessen, E. B. (2010) · Journal of Systems and Software · doi:10.1016/j.jss.2009.06.055 · Cited by 455
Nam, J., Pan, S. J., & Kim, S. (2013) · 2013 35th International Conference on Software Engineering (ICSE) · doi:10.1109/icse.2013.6606584 · Cited by 449
Self-citation
Metadata
- Authors
- Kim, S., Whitehead, & Zhang, Y.
- Venue
- IEEE Transactions on Software Engineering
- Published
- 2008-03
- Volume
- 34
- Issue
- 2
- Pages
- 181-196
- ISSN
- 0098-5589 (ISSN-L)
- 1939-3520
- 2326-3881
- OpenAlex
- W2150874999
- Cited by
- 672
- References
- 57
- FWCI
- 58.02
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.