The DOI matches its Crossref record; 54 of the 54 references it cites were found in a registry or an index.
Software Defect Prediction Using Ensemble Learning: A Systematic Literature Review
Open accessAbstract
Recent advances in the domain of software defect prediction (SDP) include the integration of multiple classification techniques to create an ensemble or hybrid approach. This technique was introduced to improve the prediction performance by overcoming the limitations of any single classification technique. This research provides a systematic literature review on the use of the ensemble learning approach for software defect prediction. The review is conducted after critically analyzing research papers published since 2012 in four well-known online libraries: ACM, IEEE, Springer Link, and Science Direct. In this study, five research questions covering the different aspects of research progress on the use of ensemble learning for software defect prediction are addressed. To extract the answers to identified questions, 46 most relevant papers are shortlisted after a thorough systematic research process. This study will provide compact information regarding the latest trends and advances in ensemble learning for software defect prediction and provide a baseline for future innovations and further reviews. Through our study, we discovered that frequently employed ensemble methods by researchers are the random forest, boosting, and bagging. Less frequently employed methods include stacking, voting and Extra Trees. Researchers proposed many promising frameworks, such as EMKCA, SMOTE-Ensemble, MKEL, SDAEsTSE, TLEL, and LRCR, using ensemble learning methods. The AUC, accuracy, F-measure, Recall, Precision, and MCC were mostly utilized to measure the prediction performance of models. WEKA was widely adopted as a platform for machine learning. Many researchers showed through empirical analysis that features selection, and data sampling was necessary pre-processing steps that improve the performance of ensemble classifiers.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Statistical pattern recognition: a review
Jain, A. K., Duin, P. W., & Mao, J. (2000) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.824819 · Cited by 6,767
Systematic Literature Reviews in Software Engineering—A Systematic Literature Review
Kitchenham, B., Pearl Brereton, O., Budgen, D., Turner, M., Bailey, J., & Linkman, S. (2009) · Information and Software Technology · doi:10.1016/j.infsof.2008.09.009 · Cited by 4,537 · cited 1×
“The systematic research process followed in this paper is based on the guidelines provided by [34]–[39].”
MethodGalar, M., Fernandez, A., Barrenechea, E., Bustince, H., & Herrera, F. (2012) · IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews) · doi:10.1109/tsmcc.2011.2161285 · Cited by 2,794
Rotation Forest: A New Classifier Ensemble Method
Rodriguez, J. J., Kuncheva, L. I., & Alonso, C. J. (2006) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2006.211 · Cited by 1,893
Optimal classifier for imbalanced data using Matthews Correlation Coefficient metric
Boughorbel, S., Jarray, F., & El-Anbari, M. (2017) · PLOS ONE · doi:10.1371/journal.pone.0177678 · Cited by 1,222 · cited 1×
“account all true and false positives and negatives to determine the quality of binary classification [14].”
BackgroundQazi, A., Hussain, F., Rahim, N. A., Hardaker, G., Alghazzawi, D., Shaban, K., & Haruna, K. (2019) · IEEE Access · doi:10.1109/access.2019.2906402 · Cited by 1,153
Ensemble approaches for regression: A survey
Mendes-Moreira, J., Soares, C., Jorge, A. M., & Sousa, J. F. D. (2012) · ACM Computing Surveys · doi:10.1145/2379776.2379786 · Cited by 667
Using Class Imbalance Learning for Software Defect Prediction
Wang, S., & Yao, X. (2013) · IEEE Transactions on Reliability · doi:10.1109/tr.2013.2259203 · Cited by 571
A systematic review of search-based testing for non-functional system properties
Afzal, W., Torkar, R., & Feldt, R. (2009) · Information and Software Technology · doi:10.1016/j.infsof.2008.12.005 · Cited by 447
A Comparison of Decision Tree Ensemble Creation Techniques
Banfield, R. E., Hall, L. O., Bowyer, K. W., & Kegelmeyer, W. P. (2007) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2007.250609 · Cited by 383
Software defect prediction using ensemble learning on selected features
Laradji, I. H., Alshayeb, M., & Ghouti, L. (2015) · Information and Software Technology · doi:10.1016/j.infsof.2014.07.005 · Cited by 380
Boosting and Other Ensemble Methods
Drucker, H., Cortes, C., Jackel, L. D., LeCun, Y., & Vapnik, V. (1994) · Neural Computation · doi:10.1162/neco.1994.6.6.1289 · Cited by 357
Deep Semantic Feature Learning for Software Defect Prediction
Wang, S., Liu, T., Nam, J., & Tan, L. (2020) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2018.2877612 · Cited by 320
Optimal combinations of pattern classifiers
Lam, L., & Suen, C. Y. (1995) · Pattern Recognition Letters · doi:10.1016/0167-8655(95)00050-q · Cited by 306
Rokach, L. (2009) · Computational Statistics & Data Analysis · doi:10.1016/j.csda.2009.07.017 · Cited by 253
TLEL: A two-layer ensemble learning approach for just-in-time defect prediction
Yang, X., Lo, D., Xia, X., & Sun, J. (2017) · Information and Software Technology · doi:10.1016/j.infsof.2017.03.007 · Cited by 216
Software defect prediction using stacked denoising autoencoders and two-stage ensemble learning
Tong, H., Liu, B., & Wang, S. (2018) · Information and Software Technology · doi:10.1016/j.infsof.2017.11.008 · Cited by 211
Software defect prediction: do different classifiers find the same defects?
Bowes, D., Hall, T., & Petrić, J. (2018) · Software Quality Journal · doi:10.1007/s11219-016-9353-3 · Cited by 207
Kittler, J., Hater, M., & Duin, R. P. W. (1996) · Proceedings of 13th International Conference on Pattern Recognition · doi:10.1109/icpr.1996.547205 · Cited by 199
Using Coding-Based Ensemble Learning to Improve Software Defect Prediction
Sun, Z., Song, Q., & Zhu, X. (2012) · IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews) · doi:10.1109/tsmcc.2012.2226152 · Cited by 187
Rapid quality assurance with Requirements Smells
Femmer, H., Méndez Fernández, D., Wagner, S., & Eder, S. (2017) · Journal of Systems and Software · doi:10.1016/j.jss.2016.02.047 · Cited by 176
Siers, M. J., & Islam, M. Z. (2015) · Information Systems · doi:10.1016/j.is.2015.02.006 · Cited by 165
Multiple kernel ensemble learning for software defect prediction
Wang, T., Zhang, Z., Jing, X., & Zhang, L. (2015) · Automated Software Engineering · doi:10.1007/s10515-015-0179-1 · Cited by 157
An Ensemble Oversampling Model for Class Imbalance Problem in Software Defect Prediction
Huda, S., Liu, K., Abdelrazek, M., Ibrahim, A. S., Alyahya, S., Al-Dossari, H. Z., & Ahmad, S. (2018) · IEEE Access · doi:10.1109/access.2018.2817572 · Cited by 155
On the Limitation of Convolutional Neural Networks in Recognizing Negative Images
Hosseini, H., Xiao, B., Jaiswal, M. S., & Poovendran, R. (2017) · International Conference on Machine Learning and Applications · doi:10.1109/icmla.2017.0-136 · Cited by 150
Tackling class overlap and imbalance problems in software defect prediction
Chen, L., Fang, B., Shang, Z., & Tang, Y. (2016) · Software Quality Journal · doi:10.1007/s11219-016-9342-6 · Cited by 133
Characterizing and predicting blocking bugs in open source projects
Valdivia-Garcia, H., & Shihab, E. (2014) · International Conference on Mining Software Repositories (MSR) · doi:10.1145/2597073.2597099 · Cited by 127
Qazi, A., Raj, R. G., Hardaker, G., & Standing, C. (2017) · Internet Research · doi:10.1108/intr-04-2016-0086 · Cited by 117
ELBlocker: Predicting blocking bugs with ensemble imbalance learning
Xia, X., Lo, D., Shihab, E., Wang, X., & Yang, X. (2015) · Information and Software Technology · doi:10.1016/j.infsof.2014.12.006 · Cited by 104
Sentiment Analysis using SVM: A Systematic Literature Review
Ahmad, M., Aftab, S., Salman, M., & Hameed, N. (2018) · International Journal of Advanced Computer Science and Applications · doi:10.14569/ijacsa.2018.090226 · Cited by 102
Self-citationTowards an ensemble based system for predicting the number of software faults
Rathore, S. S., & Kumar, S. (2017) · Expert Systems with Applications · doi:10.1016/j.eswa.2017.04.014 · Cited by 100
Rathore, S. S., & Kumar, S. (2016) · Knowledge-Based Systems · doi:10.1016/j.knosys.2016.12.017 · Cited by 93
Software defect prediction using tree-based ensembles
Aljamaan, H., & Alazba, A. (2020) · doi:10.1145/3416508.3417114 · Cited by 92
EnsembleSVM: A Library for Ensemble Learning Using Support Vector MachinesPreprint
Claesen, M., Smet, F. D., Suykens, J. A. K., & Moor, B. D. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1403.0745 · Cited by 81
An empirical study of ensemble techniques for software fault prediction
Rathore, S. S., & Kumar, S. (2020) · Applied Intelligence · doi:10.1007/s10489-020-01935-6 · Cited by 69
Evaluating and Comparing Classifiers: Review, Some Recommendations and Limitations
Stąpor, K. (2017) · Advances in intelligent systems and computing · doi:10.1007/978-3-319-59162-9_2 · Cited by 64
Yücalar, F., Özçift, A., Borandağ, E., & Kılınç, D. (2019) · Engineering Science and Technology an International Journal · doi:10.1016/j.jestch.2019.10.005 · Cited by 64
Building an Ensemble for Software Defect Prediction Based on Diversity Selection
Petrić, J., Bowes, D., Hall, T., Christianson, B., & Baddoo, N. (2016) · Proceedings of the ... ACM-IEEE International Symposium on Empirical Software Engineering and Measurement/International Symposium on Empirical Software Engineering and Measurement · doi:10.1145/2961111.2962610 · Cited by 60
The jinx on the NASA software defect data sets
Petrić, J., Bowes, D., Hall, T., Christianson, B., & Baddoo, N. (2016) · International Conference on Evaluation and Assessment in Software Engineering (EASE) · doi:10.1145/2915970.2916007 · Cited by 58
Hybrid SMOTE-Ensemble Approach for Software Defect Prediction
Alsawalqah, H., Faris, H., Aljarah, I., Alnemer, L., & Alhindawi, N. (2017) · Advances in intelligent systems and computing · doi:10.1007/978-3-319-57141-6_39 · Cited by 57
Elish, M. O., Al-Yafei, A. H., & Al-Mulhem, M. S. (2011) · Advances in Engineering Software · doi:10.1016/j.advengsoft.2011.06.001 · Cited by 54
SMOTE-Based Homogeneous Ensemble Methods for Software Defect Prediction
Balogun, A. O., Lafenwa-Balogun, F. B., Mojeed, H. A., Adeyemo, V. E., Oluwatobi, A. N., Akintola, A. G., Bajeh, A. O., & Usman-Hamza, F. E. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58817-5_45 · Cited by 52
Heterogeneous Defect Prediction Through Multiple Kernel Learning and Ensemble Learning
Li, Z., Jing, X., Zhu, X., & Zhang, H. (2017) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsme.2017.19 · Cited by 50
Alazzam, I., Alsmadi, I. M., & Akour, M. (2017) · International Journal of Data Analysis Techniques and Strategies · doi:10.1504/ijdats.2017.10003991 · Cited by 43
Latest Transformations in Scrum: A State of the Art Review
Ashraf, S., & Aftab, S. (2017) · International Journal of Modern Education and Computer Science · doi:10.5815/ijmecs.2017.07.02 · Cited by 35
Self-citationApplying Swarm Ensemble Clustering Technique for Fault Prediction Using Software Metrics
Coelho, R. A., Guimaraes, F. D. R. N., & Esmin, A. A. A. (2014) · International Conference on Machine Learning and Applications · doi:10.1109/icmla.2014.63 · Cited by 29
Ensembles based combined learning for improved software fault prediction: A comparative study
Yohannese, C. W., Li, T., Simfukwe, M., & Khurshid, F. (2017) · International Conference on Intelligent Systems and Knowledge Engineering (ISKE) · doi:10.1109/iske.2017.8258836 · Cited by 28
Performance analysis of ensemble learning for predicting defects in open source software
Kaur, A., & Kaur, K. (2014) · International Conference on Advances in Computing, Communications and Informatics (ICACCI) · doi:10.1109/icacci.2014.6968438 · Cited by 25
Latest Customizations of XP: A Systematic Literature Review
Anwer, F., & Aftab, S. (2017) · International Journal of Modern Education and Computer Science · doi:10.5815/ijmecs.2017.12.04 · Cited by 25
Self-citationScrum with the Spices of Agile Family: A Systematic Mapping
Ashraf, S. (2017) · International Journal of Modern Education and Computer Science · doi:10.5815/ijmecs.2017.11.07 · Cited by 24
A comparison of evolved linear and non-linear ensemble vote aggregators
Lacy, S., Lones, M. A., & Smith, S. L. (2015) · IEEE Congress on Evolutionary Computation (CEC) · doi:10.1109/cec.2015.7256967 · Cited by 16
A RESEARCH REVIEW ON COMPARATIVE ANALYSIS OF DATA MINING TOOLS, TECHNIQUES AND PARAMETERS
LPU, P. 1., & Sharma, A. (2017) · International Journal of Advanced Research in Computer Science · doi:10.26483/ijarcs.v8i7.4255 · Cited by 16
An Empirical Analysis on Effective Fault Prediction Model Developed Using Ensemble Methods
Kumar, L., Rath, S. K., & Sureka, A. (2017) · IEEE Annual International Computer Software and Applications Conference (COMPSAC) · doi:10.1109/compsac.2017.53 · Cited by 10
Defect Prediction of Cross Projects Using PCA and Ensemble Learning Approach
Goel, L., Sharma, M., Khatri, S. K., & Damodaran, D. (2020) · Lecture notes in networks and systems · doi:10.1007/978-981-15-2329-8_31 · Cited by 7
Cited by
Maurício, J., Domingues, I., & Bernardino, J. (2023) · Applied Sciences · doi:10.3390/app13095521 · Cited by 579
Metadata
- Authors
- Matloob, F., Ghazal, T. M., Taleb, N., Aftab, S., Ahmad, M., Khan, M. A., Abbas, S., & Soomro, T. R.
- Venue
- IEEE Access
- Published
- 2021
- Volume
- 9
- Pages
- 98754-98771
- ISSN
- 2169-3536 (ISSN-L)
- OpenAlex
- W3182306886
- Cited by
- 293
- References
- 54
- FWCI
- 9.80
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.