The DOI matches its Crossref record; 89 of the 89 references it cites were found in a registry or an index.
Heterogeneous Defect Prediction
Abstract
Many recent studies have documented the success of cross-project defect prediction (CPDP) to predict defects for new projects lacking in defect data by using prediction models built by other projects. However, most studies share the same limitations: it requires homogeneous data; i.e., different projects must describe themselves using the same metrics. This paper presents methods for heterogeneous defect prediction (HDP) that matches up different metrics in different projects. Metric matching for HDP requires a “large enough” sample of distributions in the source and target projects-which raises the question on how large is “large enough” for effective heterogeneous defect prediction. This paper shows that empirically and theoretically, “large enough” may be very small indeed. For example, using a mathematical model of defect prediction, we identify categories of data sets were as few as 50 instances are enough to build a defect prediction model. Our conclusion for this work is that, even when projects use different metric sets, it is possible to quickly transfer lessons learned about defect prediction.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Pan, S. J., & Yang, Q. (2010) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2009.191 · Cited by 23,387
The WEKA data mining software: an update
Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., & Witten, I. H. (2009) · ACM SIGKDD Explorations Newsletter · doi:10.1145/1656274.1656278 · Cited by 17,750
Individual Comparisons by Ranking Methods
Wilcoxon, F. (1945) · Biometrics Bulletin · doi:10.2307/3001968 · Cited by 15,354
An introduction to variable and feature selection
GuyonIsabelle, & ElisseeffAndré (2003) · Journal of Machine Learning Research · doi:10.5555/944919.944968 · Cited by 7,715
The Kolmogorov-Smirnov Test for Goodness of Fit
Massey, F. J. (1951) · Journal of the American Statistical Association · doi:10.1080/01621459.1951.10500769 · Cited by 7,210
An Introduction to Variable and Feature Selection.
Guyon, I., & Elisseeff, A. (2003) · Cited by 6,576
Weiss, K., Khoshgoftaar, T. M., & Wang, D. (2016) · Journal of Big Data · doi:10.1186/s40537-016-0043-6 · Cited by 6,260
McCabe, T. J. (1976) · IEEE Transactions on Software Engineering · doi:10.1109/tse.1976.233837 · Cited by 6,059
A metrics suite for object oriented design
Chidamber, S. R., & Kemerer, C. F. (1994) · IEEE Transactions on Software Engineering · doi:10.1109/32.295895 · Cited by 5,669
On the Kolmogorov-Smirnov Test for Normality with Mean and Variance Unknown
Lilliefors, H. W. (1967) · Journal of the American Statistical Association · doi:10.2307/2283970 · Cited by 4,530
Halstead, M. H. (1977) · CERN Document Server (European Organization for Nuclear Research) · Cited by 3,639
A validation of object-oriented design metrics as quality indicators
Basili, V. R., Briand, L. C., & Melo, W. L. (1996) · IEEE Transactions on Software Engineering · doi:10.1109/32.544352 · Cited by 1,726
The proof and measurement of association between two things
Spearman, C. (2010) · International Journal of Epidemiology · doi:10.1093/ije/dyq191 · Cited by 1,681
Design and code inspections to reduce errors in program development
Fagan, M. E. (1976) · IBM Systems Journal · doi:10.1147/sj.153.0182 · Cited by 1,492
Benchmarking attribute selection techniques for discrete class data mining
Hall, M. A., & Holmes, G. (2003) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2003.1245283 · Cited by 1,232
Lessmann, S., Baesens, B., Mues, C., & Pietsch, S. (2008) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2008.35 · Cited by 1,226
Data Mining Static Code Attributes to Learn Defect Predictors
Menzies, T., Greenwald, J., & Frank, A. (2007) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2007.256941 · Cited by 1,215
Self-citationLandwehr, N., Hall, M., & Frank, E. (2005) · Machine Learning · doi:10.1007/s10994-005-0466-3 · Cited by 1,214
Arcuri, A., & Briand, L. (2011) · Proceedings of the 33rd International Conference on Software Engineering · doi:10.1145/1985793.1985795 · Cited by 968
Nonparametric Statistics for Non‐Statisticians: A Step‐by‐Step Approach
Corder, G. W., & Foreman, D. I. (2009) · Wiley eBooks · doi:10.1002/9781118165881 · Cited by 954
Mining metrics to predict component failures
Nagappan, N., Ball, T., & Zeller, A. (2006) · Proceedings of the 28th international conference on Software engineering · doi:10.1145/1134285.1134349 · Cited by 782
On the relative value of cross-company and within-company data for defect prediction
Turhan, B., Menzies, T., Bener, A. B., & Di Stefano, J. (2009) · Empirical Software Engineering · doi:10.1007/s10664-008-9103-7 · Cited by 725
Self-citationPredicting the location and number of faults in large software systems
Ostrand, T. J., Weyuker, E. J., & Bell, R. M. (2005) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2005.49 · Cited by 720
A large-scale empirical study of just-in-time quality assurance
Kamei, Y., Shihab, E., Adams, B., Hassan, A. E., Mockus, A., Sinha, A., & Ubayashi, N. (2013) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2012.70 · Cited by 715
Cross-project defect prediction: a large scale experiment on data vs. domain vs. process
Zimmermann, T., Nagappan, N., Gall, H. C., Giger, E., & Murphy, B. (2009) · Zurich Open Repository and Archive (University of Zurich) · doi:10.5167/uzh-25785 · Cited by 694
The PROMISE Repository of Empirical Software Engineering Data
Boetticher, G. D. (2007) · Medical Entomology and Zoology · Cited by 682
Evaluating defect prediction approaches: a benchmark and an extensive comparison
D’Ambros, M., Lanza, M., & Robbes, R. (2012) · Empirical Software Engineering · doi:10.1007/s10664-011-9173-9 · Cited by 626
An Empirical Comparison of Model Validation Techniques for Defect Prediction Models
Tantithamthavorn, C., McIntosh, S., Hassan, A. E., & Matsumoto, K. (2017) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2016.2584050 · Cited by 574
Data Quality: Some Comments on the NASA Software Defect Datasets
Shepperd, M., Song, Q., Sun, Z., & Mair, C. (2013) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2013.11 · Cited by 570
Transfer learning for cross-company software defect prediction
Ma, Y., Luo, G., Zeng, X., & Chen, A. (2012) · Information and Software Technology · doi:10.1016/j.infsof.2011.09.007 · Cited by 475
Nam, J., Pan, S. J., & Kim, S. (2013) · 2013 35th International Conference on Software Engineering (ICSE) · doi:10.1109/icse.2013.6606584 · Cited by 449
Self-citationReLink: recovering links between bugs and changes
Wu, R., Zhang, H., Kim, S., & Cheung, S.-C. (2011) · Proceedings of the 19th ACM SIGSOFT symposium and the 13th European conference on Foundations of software engineering · doi:10.1145/2025113.2025120 · Cited by 412
Self-citationLiu, H., Li, J., & Wong, L. (2002) · PubMed · Cited by 410
A General Software Defect-Proneness Prediction Framework
Song, Q., Jia, Z., Shepperd, M., Ying, S., & Liu, J. (2011) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2010.90 · Cited by 374
Revisiting the Impact of Classification Techniques on the Performance of Defect Prediction Models
Ghotra, B., McIntosh, S., & Hassan, A. E. (2015) · 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering · doi:10.1109/icse.2015.91 · Cited by 364
Predicting fault-prone software modules in telephone switches
Ohlsson, N., & Alberg, H. (1996) · IEEE Transactions on Software Engineering · doi:10.1109/32.553637 · Cited by 346
Catal, C., & Diri, B. (2009) · Information Sciences · doi:10.1016/j.ins.2008.12.001 · Cited by 343
An investigation on the feasibility of cross-project defect prediction
He, Z., Shu, F., Yang, Y., Li, M., & Wang, Q. (2012) · Automated Software Engineering · doi:10.1007/s10515-011-0090-3 · Cited by 333
Facts and fallacies of software engineering
Glass, R. L. (2002) · Cited by 332
Automated parameter optimization of classification techniques for defect prediction models
Tantithamthavorn, C., McIntosh, S., Hassan, A. E., & Matsumoto, K. (2016) · Proceedings of the 38th International Conference on Software Engineering · doi:10.1145/2884781.2884857 · Cited by 331
Software testability: the new verification
Voas, J. M., & Miller, K. W. (1995) · IEEE Software · doi:10.1109/52.382180 · Cited by 327
Understanding and Using Linear Programming
(2007) · Universitext · doi:10.1007/978-3-540-30717-4 · Cited by 326
An empirical study on software defect prediction with a simplified metric set
He, P., Li, B., Liu, X., Chen, J., & Ma, Y. (2015) · Information and Software Technology · doi:10.1016/j.infsof.2014.11.006 · Cited by 309
Choosing software metrics for defect prediction: an investigation on feature selection techniques
Gao, K., Khoshgoftaar, T. M., Wang, H., & Seliya, N. (2011) · Software: Practice and Experience · doi:10.1002/spe.1043 · Cited by 281
Static analysis tools as early indicators of pre-release defect density
Nagappan, N., & Ball, T. (2005) · Proceedings of the 27th international conference on Software engineering - ICSE '05 · doi:10.1145/1062455.1062558 · Cited by 272
Local versus Global Lessons for Defect Prediction and Effort Estimation
Menzies, T., Butcher, A., Cok, D., Marcus, A., Layman, L., Shull, F., Turhan, B., & Zimmermann, T. (2013) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2012.83 · Cited by 261
Self-citationCan developer-module networks predict failures?
Pinzger, M., Nagappan, N., & Murphy, B. (2008) · Proceedings of the 16th ACM SIGSOFT International Symposium on Foundations of software engineering · doi:10.1145/1453101.1453105 · Cited by 256
How, and why, process metrics are better
Rahman, F., & Devanbu, P. (2013) · 2013 35th International Conference on Software Engineering (ICSE) · doi:10.1109/icse.2013.6606589 · Cited by 248
Reducing Features to Improve Code Change-Based Bug Prediction
Shivaji, S., Whitehead, E. J., Akella, R., & Kim, S. (2013) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2012.43 · Cited by 247
Self-citationRecalling the "imprecision" of cross-project defect prediction
Rahman, F., Posnett, D., & Devanbu, P. (2012) · Proceedings of the ACM SIGSOFT 20th International Symposium on the Foundations of Software Engineering · doi:10.1145/2393596.2393669 · Cited by 246
Cross-project defect prediction using a connectivity-based unsupervised classifier
Zhang, F., Zheng, Q., Zou, Y., & Hassan, A. E. (2016) · Proceedings of the 38th International Conference on Software Engineering · doi:10.1145/2884781.2884839 · Cited by 245
What we have learned about fighting defects
Shull, F., Basili, V., Boehm, B., Brown, A. W., Costa, P., Lindvall, M., Port, D., Rus, I., Tesoriero, R., & Zelkowitz, M. (2002) · Proceedings Eighth IEEE Symposium on Software Metrics · doi:10.1109/metric.2002.1011343 · Cited by 244
Yang, Y., Zhou, Y., Liu, J., Zhao, Y., Lu, H., Xu, L., Xu, B., & Leung, H. (2016) · Proceedings of the 2016 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering · doi:10.1145/2950290.2950353 · Cited by 229
Tuning for software analytics: Is it really necessary?
Fu, W., Menzies, T., & Shen, X. (2016) · Information and Software Technology · doi:10.1016/j.infsof.2016.04.017 · Cited by 210
Self-citationSample-based software defect prediction with active and semi-supervised learning
Li, M., Zhang, H., Wu, R., & Zhou, Z. (2011) · Automated Software Engineering · doi:10.1007/s10515-011-0092-1 · Cited by 199
Jing, X.-Y., Wu, F., Dong, X., & Xu, B. (2017) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2016.2597849 · Cited by 198
Jing, X., Wu, F., Dong, X., Qi, F., & Xu, B. (2015) · Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering · doi:10.1145/2786805.2786813 · Cited by 197
CLAMI: Defect Prediction on Unlabeled Datasets (T)
Nam, J., & Kim, S. (2015) · 2015 30th IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase.2015.56 · Cited by 192
Self-citationA Handbook of Software and Systems Engineering: Empirical Observations, Laws and Theories
Endres, A., & Rombach, H. D. (2003) · Cited by 185
Where Should We Fix This Bug? A Two-Phase Recommendation Model
Kim, D., Tao, Y., Kim, S., & Zeller, A. (2013) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2013.24 · Cited by 185
Self-citationHeterogeneous defect prediction
Nam, J., & Kim, S. (2015) · Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering · doi:10.1145/2786805.2786814 · Cited by 182
Self-citationAn empirical study of just-in-time defect prediction using cross-project models
Fukushima, T., Kamei, Y., McIntosh, S., Yamashita, K., & Ubayashi, N. (2014) · Proceedings of the 11th Working Conference on Mining Software Repositories · doi:10.1145/2597073.2597075 · Cited by 180
Cross-project defect prediction models: L'Union fait la force
Panichella, A., Oliveto, R., & De Lucia, A. (2014) · 2014 Software Evolution Week - IEEE Conference on Software Maintenance, Reengineering, and Reverse Engineering (CSMR-WCRE) · doi:10.1109/csmr-wcre.2014.6747166 · Cited by 176
Value-cognitive boosting with a support vector machine for cross-project defect prediction
Ryu, D., Choi, O., & Baik, J. (2016) · Empirical Software Engineering · doi:10.1007/s10664-014-9346-4 · Cited by 176
Multi-objective Cross-Project Defect Prediction
Canfora, G., De Lucia, A., Di Penta, M., Oliveto, R., Panichella, A., & Panichella, S. (2013) · 2013 IEEE Sixth International Conference on Software Testing, Verification and Validation · doi:10.1109/icst.2013.38 · Cited by 171
Micro interaction metrics for defect prediction
Lee, T., Nam, J., Han, D., Kim, S., & In, H. P. (2011) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/2025113.2025156 · Cited by 167
Self-citationGiger, E., D'Ambros, M., Pinzger, M., & Gall, H. C. (2012) · Proceedings of the ACM-IEEE international symposium on Empirical software engineering and measurement · doi:10.1145/2372251.2372285 · Cited by 164
A transfer cost-sensitive boosting approach for cross-project defect prediction
Ryu, D., Jang, J., & Baik, J. (2015) · Software Quality Journal · doi:10.1007/s11219-015-9287-1 · Cited by 159
Does bug prediction support human developers? Findings from a Google case study
Lewis, C., Lin, Z., Sadowski, C., Zhu, X., Ou, R., & Whitehead, E. J. (2013) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse.2013.6606583 · Cited by 148
On the dataset shift problem in software engineering prediction models
Turhan, B. (2011) · Empirical Software Engineering · doi:10.1007/s10664-011-9182-8 · Cited by 132
Comparing static bug finders and statistical prediction
Rahman, F., Khatri, S., Barr, E. T., & Dévanbu, P. (2014) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/2568225.2568269 · Cited by 121
Improving classification performance with discretization on biomedical datasets.
Lustgarten, J. L., Gopalakrishnan, V., Grover, H., & Visweswaran, S. (2008) · PubMed · Cited by 116
Kocagüneli, E., Menzies, T., Keung, J., Cok, D. R., & Madachy, R. J. (2013) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2012.88 · Cited by 108
Self-citationShepperd, M., & Ince, D. C. (1994) · Journal of Systems and Software · doi:10.1016/0164-1212(94)90011-6 · Cited by 107
Software verification and validation for practitioners and managers
Rakitin, S. R. (2001) · Cited by 100
Ecological inference in empirical software engineering
Posnett, D., Filkov, V., & Dévanbu, P. (2011) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase.2011.6100074 · Cited by 83
AI‐Based Software Defect Predictors: Applications and Benefits in a Case Study
Tosun, A., Bener, A., & Kale, R. (2011) · AI Magazine · doi:10.1609/aimag.v32i2.2348 · Cited by 73
Privacy and utility for defect prediction: Experiments with MORPH
Peters, F., & Menzies, T. (2012) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse.2012.6227194 · Cited by 69
Self-citationToo much automation? the bellwether effect and its implications for transfer learning
Krishna, R., Menzies, T., & Fu, W. (2016) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/2970276.2970339 · Cited by 68
Self-citationAutomatically Generating Test Cases for Specification Mining
Dallmeier, V., Knopp, N., Mallon, C., Fraser, G., Hack, S., & Zeller, A. (2012) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2011.105 · Cited by 67
Tosun, A., Turhan, B., & Bener, A. (2009) · doi:10.1145/1540438.1540453 · Cited by 65
Replication of defect prediction studies
Mende, T. (2010) · doi:10.1145/1868328.1868336 · Cited by 58
Towards Cross-Project Defect Prediction with Imbalanced Feature SetsPreprint
He, P., Li, B., & Ma, Y. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1411.4228 · Cited by 54
Towards improving statistical modeling of software engineering data: think locally, act globally!
Bettenburg, N., Nagappan, M., & Hassan, A. E. (2014) · Empirical Software Engineering · doi:10.1007/s10664-013-9292-6 · Cited by 38
Menzies, T., Raffo, D., Setamanit, S., Hu, Y., & Tootoonian, S. (2002) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase.2002.1115012 · Cited by 35
Self-citationREMI: defect prediction for efficient API testing
Kim, M., Nam, J., Yeon, J., Choi, S., & Kim, S. (2015) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/2786805.2804429 · Cited by 33
Self-citationKläs, M., Elberzhager, F., Münch, J., Hartjes, K., & von Graevemeyer, O. (2010) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/1810295.1810313 · Cited by 29
Sigweni, B., & Shepperd, M. (2014) · doi:10.1145/2639490.2639508 · Cited by 21
Menzies, T. (2013) · Springer eBooks · doi:10.1007/978-3-642-45135-5_3 · Cited by 5
Self-citation
Cited by
Analysis of transfer learning for deep neural network based plant classification models
Kaya, A., Keceli, A. S., Catal, C., Yalic, H. Y., Temucin, H., & Tekinerdogan, B. (2019) · Computers and Electronics in Agriculture · doi:10.1016/j.compag.2019.01.041 · Cited by 440
Deep Semantic Feature Learning for Software Defect Prediction
Wang, S., Liu, T., Nam, J., & Tan, L. (2020) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2018.2877612 · Cited by 320
Self-citationA Systematic Review on Data Scarcity Problem in Deep Learning: Solution and Applications
Bansal, M. A., Sharma, D. R., & Kathuria, D. M. (2022) · ACM Computing Surveys · doi:10.1145/3502287 · Cited by 306
An Empirical Study of Model-Agnostic Techniques for Defect Prediction Models
Jiarpakdee, J., Tantithamthavorn, C. K., Dam, H. K., & Grundy, J. (2022) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2020.2982385 · Cited by 148
Metadata
- Authors
- Nam, J., Fu, W., Kim, S., Menzies, T., & Tan, L.
- Venue
- IEEE Transactions on Software Engineering
- Published
- 2018-09-01
- Volume
- 44
- Issue
- 9
- Pages
- 874-896
- ISSN
- 0098-5589 (ISSN-L)
- 1939-3520
- 2326-3881
- OpenAlex
- W2731935965
- Cited by
- 277
- References
- 89
- FWCI
- 52.47
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.