The DOI matches its Crossref record; 64 of the 64 references it cites were found in a registry or an index.
An empirical study on software defect prediction with a simplified metric set
Open accessAbstract
ContextSoftware defect prediction plays a crucial role in estimating the most defect-prone components of software, and a large number of studies have pursued improving prediction accuracy within a project or across projects. However, the rules for making an appropriate decision between within- and cross-project defect prediction when available historical data are insufficient remain unclear. ObjectiveThe objective of this work is to validate the feasibility of the predictor built with a simplified metric set for software defect prediction in different scenarios, and to investigate practical guidelines for the choice of training data, classifier and metric subset of a given project. MethodFirst, based on six typical classifiers, three types of predictors using the size of software metric set were constructed in three scenarios. Then, we validated the acceptable performance of the predictor based on Top-k metrics in terms of statistical methods. Finally, we attempted to minimize the Top-k metric subset by removing redundant metrics, and we tested the stability of such a minimum metric subset with one-way ANOVA tests. ResultsThe study has been conducted on 34 releases of 10 open-source projects available at the PROMISE repository. The findings indicate that the predictors built with either Top-k metrics or the minimum metric subset can provide an acceptable result compared with benchmark predictors. The guideline for choosing a suitable simplified metric set in different scenarios is presented in Table 12. ConclusionThe experimental results indicate that (1) the choice of training data for defect prediction should depend on the specific requirement of accuracy; (2) the predictor built with a simplified metric set works well and is very useful in case limited resources are supplied; (3) simple classifiers (e.g., Naive Bayes) also tend to perform well when using a simplified metric set for defect prediction; and (4) in several cases, the minimum metric subset can be identified to facilitate the procedure of general defect prediction with acceptable loss of prediction precision in practice.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Book Review: Pattern Recognition and Machine Learning
Nasrabadi, N. M. (2007) · Journal of Electronic Imaging · doi:10.1117/1.2819119 · Cited by 22,066
An introduction to ROC analysis
Fawcett, T. (2006) · Pattern Recognition Letters · doi:10.1016/j.patrec.2005.10.010 · Cited by 21,879
Peng, H., Long, F., & Ding, C. (2005) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2005.159 · Cited by 10,544
(2006) · Springer eBooks · Cited by 8,123
An empirical study of the naive Bayes classifier
Rish, I. (2001) · Cited by 3,202
Toward integrating feature selection algorithms for classification and clustering
Liu, H., & Yu, L. (2005) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2005.66 · Cited by 2,706
Pattern Recognition and Machine Learning
Maindonald, J. (2007) · Journal of Statistical Software · doi:10.18637/jss.v017.b05 · Cited by 1,349
Lessmann, S., Baesens, B., Mues, C., & Pietsch, S. (2008) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2008.35 · Cited by 1,226
Data Mining Static Code Attributes to Learn Defect Predictors
Menzies, T., Greenwald, J., & Frank, A. (2007) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2007.256941 · Cited by 1,215
A Systematic Literature Review on Fault Prediction Performance in Software Engineering
Hall, T., Beecham, S., Bowes, D., Gray, D., & Counsell, S. (2012) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2011.103 · Cited by 1,154
Empirical validation of object-oriented metrics on open source software for fault prediction
Gyimothy, T., Ferenc, R., & Siket, I. (2005) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2005.112 · Cited by 904
On the relative value of cross-company and within-company data for defect prediction
Turhan, B., Menzies, T., Bener, A. B., & Di Stefano, J. (2009) · Empirical Software Engineering · doi:10.1007/s10664-008-9103-7 · Cited by 725
Kohavi, R. (1995) · Lecture Notes in Computer Science · doi:10.1007/3-540-59286-5_57 · Cited by 712
Quantitative analysis of faults and failures in a complex software system
Fenton, N. E., & Ohlsson, N. (2000) · IEEE Transactions on Software Engineering · doi:10.1109/32.879815 · Cited by 699
Cross-project defect prediction: a large scale experiment on data vs. domain vs. process
Zimmermann, T., Nagappan, N., Gall, H. C., Giger, E., & Murphy, B. (2009) · Zurich Open Repository and Archive (University of Zurich) · doi:10.5167/uzh-25785 · Cited by 694
Subramanyam, R., & Krishnan, M. S. (2003) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2003.1191795 · Cited by 672
Evaluating defect prediction approaches: a benchmark and an extensive comparison
D’Ambros, M., Lanza, M., & Robbes, R. (2012) · Empirical Software Engineering · doi:10.1007/s10664-011-9173-9 · Cited by 626
Shin, Y., Meneely, A., Williams, L., & Osborne, J. A. (2011) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2010.81 · Cited by 607
Predicting defects using network analysis on dependency graphs
Zimmermann, T., & Nagappan, N. (2008) · Proceedings of the 13th international conference on Software engineering - ICSE '08 · doi:10.1145/1368088.1368161 · Cited by 580
Towards identifying software project clusters with regard to defect prediction
Jureczko, M., & Madeyski, L. (2010) · Proceedings of the 6th International Conference on Predictive Models in Software Engineering · doi:10.1145/1868328.1868342 · Cited by 546
Software fault prediction metrics: A systematic literature review
Radjenović, D., Heričko, M., Torkar, R., & Živkovič, A. (2013) · Information and Software Technology · doi:10.1016/j.infsof.2013.02.009 · Cited by 538
Transfer learning for cross-company software defect prediction
Ma, Y., Luo, G., Zeng, X., & Chen, A. (2012) · Information and Software Technology · doi:10.1016/j.infsof.2011.09.007 · Cited by 475
Self-citationDefect prediction from static code features: current results, limitations, new approaches
Menzies, T., Milton, Z., Turhan, B., Cukic, B., Jiang, Y., & Bener, A. (2010) · Automated Software Engineering · doi:10.1007/s10515-010-0069-5 · Cited by 466
Arisholm, E., Briand, L. C., & Johannessen, E. B. (2010) · Journal of Systems and Software · doi:10.1016/j.jss.2009.06.055 · Cited by 455
Nam, J., Pan, S. J., & Kim, S. (2013) · 2013 35th International Conference on Software Engineering (ICSE) · doi:10.1109/icse.2013.6606584 · Cited by 449
Cliff´s Delta Calculator: A non-parametric effect size program for two groups of observations
Macbeth, G., Razumiejczyk, E., & Ledesma, R. D. (2010) · Universitas Psychologica · doi:10.11144/javeriana.upsy10-2.cdcp · Cited by 442
ReLink: recovering links between bugs and changes
Wu, R., Zhang, H., Kim, S., & Cheung, S.-C. (2011) · Proceedings of the 19th ACM SIGSOFT symposium and the 13th European conference on Foundations of software engineering · doi:10.1145/2025113.2025120 · Cited by 412
A General Software Defect-Proneness Prediction Framework
Song, Q., Jia, Z., Shepperd, M., Ying, S., & Liu, J. (2011) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2010.90 · Cited by 374
A stability index for feature selection
Kuncheva, L. I. (2007) · Conference on Artificial Intelligence for Applications · Cited by 354
Assessing the applicability of fault-proneness models across object-oriented software projects
Briand, L. C., Melo, W. L., & Wust, J. (2002) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2002.1019484 · Cited by 352
Empirical Analysis of Object-Oriented Design Metrics for Predicting High and Low Severity Faults
Zhou, Y., & Leung, H. (2006) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2006.102 · Cited by 348
Catal, C., & Diri, B. (2009) · Information Sciences · doi:10.1016/j.ins.2008.12.001 · Cited by 343
Software fault prediction: A literature review and current trends
Catal, C. (2011) · Expert Systems with Applications · doi:10.1016/j.eswa.2010.10.024 · Cited by 336
An investigation on the feasibility of cross-project defect prediction
He, Z., Shu, F., Yang, Y., Li, M., & Wang, Q. (2012) · Automated Software Engineering · doi:10.1007/s10515-011-0090-3 · Cited by 333
Software defect prediction using Bayesian networks
Okutan, A., & Yıldız, O. T. (2014) · Empirical Software Engineering · doi:10.1007/s10664-012-9218-8 · Cited by 330
Nam, J., Pan, S. J., & Kim, S. (2014) · Cited by 282
Can developer-module networks predict failures?
Pinzger, M., Nagappan, N., & Murphy, B. (2008) · Proceedings of the 16th ACM SIGSOFT International Symposium on Foundations of software engineering · doi:10.1145/1453101.1453105 · Cited by 256
Reducing Features to Improve Code Change-Based Bug Prediction
Shivaji, S., Whitehead, E. J., Akella, R., & Kim, S. (2013) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2012.43 · Cited by 247
Recalling the "imprecision" of cross-project defect prediction
Rahman, F., Posnett, D., & Devanbu, P. (2012) · Proceedings of the ACM SIGSOFT 20th International Symposium on the Foundations of Software Engineering · doi:10.1145/2393596.2393669 · Cited by 246
The distribution of faults in a large industrial software system
Ostrand, T. J., & Weyuker, E. J. (2002) · ACM SIGSOFT Software Engineering Notes · doi:10.1145/566171.566181 · Cited by 246
Predicting failures with developer networks and social network analysis
Meneely, A., Williams, L., Snipes, W., & Osborne, J. (2008) · Proceedings of the 16th ACM SIGSOFT International Symposium on Foundations of software engineering · doi:10.1145/1453101.1453106 · Cited by 245
Personalized defect prediction
Jiang, T., Tan, L., & Kim, S. (2013) · 2013 28th IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase.2013.6693087 · Cited by 240
Predicting build failures using social network analysis on developer communication
Wolf, T., Schroter, A., Damian, D., & Nguyen, T. (2009) · 2009 IEEE 31st International Conference on Software Engineering · doi:10.1109/icse.2009.5070503 · Cited by 224
Better cross company defect prediction
Peters, F., Menzies, T., & Marcus, A. (2013) · 2013 10th Working Conference on Mining Software Repositories (MSR) · doi:10.1109/msr.2013.6624057 · Cited by 214
Software defect association mining and defect correction effort prediction
Song, Q., Shepperd, M., Cartwright, M., & Mair, C. (2006) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2006.1599417 · Cited by 205
Intermediate Statistical Methods and Applications: A Computer Package Approach
Berenson, M. L., Levine, D. M., & Goldstein, M. (1982) · Andalas University Repository (Andalas University) · Cited by 194
The missing links: bugs and bug-fix commits
Bachmann, A., Bird, C., Rahman, F., Devanbu, P., & Bernstein, A. (2010) · Proceedings of the eighteenth ACM SIGSOFT international symposium on Foundations of software engineering · doi:10.1145/1882291.1882308 · Cited by 190
Using Software Dependencies and Churn Metrics to Predict Field Failures: An Empirical Case Study
Nagappan, N., & Ball, T. (2007) · First International Symposium on Empirical Software Engineering and Measurement (ESEM 2007) · doi:10.1109/esem.2007.13 · Cited by 176
(2007) · First International Symposium on Empirical Software Engineering and Measurement (ESEM 2007) · doi:10.1109/esem.2007.87 · Cited by 170
Training data selection for cross-project defect prediction
Herbold, S. (2013) · doi:10.1145/2499393.2499395 · Cited by 158
An investigation of the relationships between lines of code and defects
Zhang, H. (2009) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsm.2009.5306304 · Cited by 158
Graph-based analysis and prediction for software evolution
Bhattacharya, P., Iliofotou, M., Neamtiu, I., & Faloutsos, M. (2012) · doi:10.5555/2337223.2337273 · Cited by 151
Intermediate Statistical Methods and Applications: A Computer Package Approach.
Weisberg, S., Berenson, M. L., Levine, D. M., Goldstein, M., Cooper, R. A., & Weekes, A. J. (1984) · Journal of the American Statistical Association · doi:10.2307/2288297 · Cited by 144
A Second Replicated Quantitative Analysis of Fault Distributions in Complex Software Systems
Grbac, T. G., Runeson, P., & Huljenić, D. (2012) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2012.46 · Cited by 138
Graph-based analysis and prediction for software evolution
Bhattacharya, P., Iliofotou, M., Neamtiu, I., & Faloutsos, M. (2012) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse.2012.6227173 · Cited by 138
A Replicated Quantitative Analysis of Fault Distributions in Complex Software Systems
Andersson, C., & Runeson, P. (2007) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2007.1005 · Cited by 135
Analysis of Naive Bayes’ assumptions on software fault data: An empirical study
Turhan, B., & Bener, A. (2008) · Data & Knowledge Engineering · doi:10.1016/j.datak.2008.10.005 · Cited by 133
Empirical evaluation of the effects of mixed project data on learning defect predictors
Turhan, B., Tosun, A., & Bener, A. (2012) · Information and Software Technology · doi:10.1016/j.infsof.2012.10.003 · Cited by 120
Validation of network measures as indicators of defective modules in software systems
Tosun, A., Turhan, B., & Bener, A. (2009) · doi:10.1145/1540438.1540446 · Cited by 90
Network Versus Code Metrics to Predict Defects: A Replication Study
Premraj, R., & Herzig, K. (2011) · Proceedings of the ... ACM-IEEE International Symposium on Empirical Software Engineering and Measurement/International Symposium on Empirical Software Engineering and Measurement · doi:10.1109/esem.2011.30 · Cited by 82
Jin, C., & Liu, J.-A. (2010) · International Conference on Multimedia and Information Technology · doi:10.1109/mmit.2010.10 · Cited by 49
A Hybrid Set of Complexity Metrics for Large-Scale Object-Oriented Software Systems
Ma, Y., He, K., Li, B., Liu, J., & Zhou, X.-Y. (2010) · Journal of Computer Science and Technology · doi:10.1007/s11390-010-9398-x · Cited by 48
Self-citationEnsemble of software defect predictors
Tosun, A., Turhan, B., & Bener, A. (2008) · Proceedings of the ... ACM-IEEE International Symposium on Empirical Software Engineering and Measurement/International Symposium on Empirical Software Engineering and Measurement · doi:10.1145/1414004.1414066 · Cited by 40
An Empirical Study on the Relation between Dependency Neighborhoods and Failures
Zimmerman, T., Nagappan, N., Herzig, K., Premraj, R., & Williams, L. A. (2011) · IEEE International Conference on Software Testing, Verification and Validation (ICST) · doi:10.1109/icst.2011.39 · Cited by 32
Metadata
- Authors
- He, P., Li, B., Liu, X., Chen, J., & Ma, Y.
- Venue
- Information and Software Technology
- Publisher
- Elsevier BV
- Published
- 2015-03
- Volume
- 59
- Pages
- 170-190
- ISSN
- 0950-5849 (ISSN-L)
- OpenAlex
- W2099919734
- Cited by
- 309
- References
- 64
- FWCI
- 46.23
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.