The DOI matches its Crossref record; 151 of the 151 references it cites were found in a registry or an index.
The Impact of Automated Parameter Optimization on Defect Prediction Models
Open accessAbstract
Defect prediction models-classifiers that identify defect-prone software modules-have configurable parameters that control their characteristics (e.g., the number of trees in a random forest). Recent studies show that these classifiers underperform when default settings are used. In this paper, we study the impact of automated parameter optimization on defect prediction models. Through a case study of 18 datasets, we find that automated parameter optimization: (1) improves AUC performance by up to 40 percentage points; (2) yields classifiers that are at least as stable as those trained using default settings; (3) substantially shifts the importance ranking of variables, with as few as 28 percent of the top-ranked variables in optimized classifiers also being top-ranked in non-optimized classifiers; (4) yields optimized settings for 17 of the 20 most sensitive parameters that transfer among datasets without a statistically significant drop in performance; and (5) adds less than 30 minutes of additional computation to 12 of the 26 studied classification techniques. While widely-used classification techniques like random forest and support vector machines are not optimization-sensitive, traditionally overlooked techniques like C5.0 and neural networks can actually outperform widely-used techniques after optimization is applied. This highlights the importance of exploring the parameter space when using parameter-sensitive classification techniques.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
R: A Language and Environment for Statistical Computing
R Core Team (2025) · doi:10.32614/r.manuals · Cited by 352,181
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538
Statistical Power Analysis for the Behavioral Sciences
Muller, K. (1989) · Technometrics · doi:10.1080/00401706.1989.10488618 · Cited by 99,167
Scikit-learn: Machine Learning in Python
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Müller, A., Nothman, J., Louppe, G., et al. (2012) · arXiv · doi:10.48550/arxiv.1201.0490 · Cited by 71,016
Statistical power analysis for the behavioral sciences
(1990) · Computers, Environment and Urban Systems · doi:10.1016/0198-9715(90)90050-4 · Cited by 65,269
An Introduction to the Bootstrap
Efron, B., & Tibshirani, R. J. (1994) · doi:10.1201/9780429246593 · Cited by 40,500
Storn, R., & Price, K. (1997) · Journal of Global Optimization · doi:10.1023/a:1008202821328 · Cited by 29,569
An Introduction to the Bootstrap
Efron, B., & Tibshirani, R. J. (1993) · doi:10.1007/978-1-4899-4541-9 · Cited by 21,230
The WEKA data mining software: an update
Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., & Witten, I. H. (2009) · ACM SIGKDD Explorations Newsletter · doi:10.1145/1656274.1656278 · Cited by 17,750 · cited 5×
“For example, for the number of decision trees for a random forest classifier, the default varies among settings of 10 for the bigrf R package [82], 50 for MATLAB [88], 100 for Weka [43], and 500 for the randomForest R package [81].”
MethodInfluentialTHE USE OF MULTIPLE MEASUREMENTS IN TAXONOMIC PROBLEMS
FISHER, R. A. (1936) · Annals of Eugenics · doi:10.1111/j.1469-1809.1936.tb02137.x · Cited by 15,227
He, H., & Garcia, E. A. (2009) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2008.239 · Cited by 10,359
A simulation study of the number of events per variable in logistic regression analysis
Peduzzi, P., Concato, J., Kemper, E., Holford, T. R., & Feinstein, A. R. (1996) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(96)00236-3 · Cited by 9,655
Building Predictive Models in R Using the caret Package
Kuhn, M. (2008) · Journal of Statistical Software · doi:10.18637/jss.v028.i05 · Cited by 8,863
Cohen, J. (1992) · Psychological Bulletin · doi:10.1037//0033-2909.112.1.155 · Cited by 8,771
(2012) · Cited by 8,009
Multivariate Adaptive Regression Splines
Friedman, J. H. (1991) · The Annals of Statistics · doi:10.1214/aos/1176347963 · Cited by 8,000
Harrell (2015) · Springer Series in Statistics · doi:10.1007/978-3-319-19425-7 · Cited by 6,878
The relationship between Precision-Recall and ROC curves
Davis, J., & Goadrich, M. (2006) · Proceedings of the 23rd international conference on Machine learning - ICML '06 · doi:10.1145/1143844.1143874 · Cited by 6,559
Tenenhaus, M., Vinzi, V. E., Chatelin, Y.-M., & Lauro, C. (2005) · Computational Statistics & Data Analysis · doi:10.1016/j.csda.2004.03.005 · Cited by 6,312
Supervised Machine Learning: A Review of Classification Techniques
Kotsiantis, S. B. (2007) · Informatica · Cited by 6,008
Assessing the Performance of Prediction Models: A Framework for Traditional and Novel Measures
Steyerberg, E. W., Vickers, A. J., Cook, N. R., Gerds, T., Gonen, M., Obuchowski, N., Pencina, M. J., & Kattan, M. W. (2010) · Epidemiology · doi:10.1097/ede.0b013e3181c30fb2 · Cited by 5,170
`` Direct Search'' Solution of Numerical and Statistical Problems
Hooke, R., & Jeeves, T. A. (1961) · Journal of the ACM · doi:10.1145/321062.321069 · Cited by 4,254
Clinical Prediction Models: A Practical Approach to Development, Validation, and Updating
Steyerberg, E. W. (2019) · Statistics for Biology and Health · doi:10.1007/978-3-030-16399-0 · Cited by 3,281
Genetic Algorithms and Machine Learning
Goldberg, D. E., & Holland, J. H. (1988) · Machine Learning · doi:10.1023/a:1022602019183 · Cited by 3,193
Estimating the error rate of a prediction rule: improvement on cross-validation
Efron, B. (1983) · Journal of the American Statistical Association · doi:10.1080/01621459.1983.10477973 · Cited by 2,300
Using AUC and accuracy in evaluating learning algorithms
Huang, J., & Ling, C. X. (2005) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2005.50 · Cited by 2,148
Very Simple Classification Rules Perform Well on Most Commonly Used Datasets
Holte, R. C. (1993) · Machine Learning · doi:10.1023/a:1022631118932 · Cited by 1,791
caret: Classification and Regression Training
Kuhn, M. (2007) · CRAN: Contributed Packages · doi:10.32614/cran.package.caret · Cited by 1,786
Graph-based approach for airborne light detection and ranging segmentation
Vilariño, D. L., Cabaleiro, J. C., Martínez, J., Rivera, F. F., & Pena, T. F. (2017) · Journal of Applied Remote Sensing · doi:10.1117/1.jrs.11.015020 · Cited by 1,721
Genetic Algorithms and Random Keys for Sequencing and Optimization
Bean, J. C. (1994) · INFORMS journal on computing · doi:10.1287/ijoc.6.2.154 · Cited by 1,543
Lessmann, S., Baesens, B., Mues, C., & Pietsch, S. (2008) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2008.35 · Cited by 1,226
Data Mining Static Code Attributes to Learn Defect Predictors
Menzies, T., Greenwald, J., & Frank, A. (2007) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2007.256941 · Cited by 1,215
A Systematic Literature Review on Fault Prediction Performance in Software Engineering
Hall, T., Beecham, S., Bowes, D., Gray, D., & Counsell, S. (2012) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2011.103 · Cited by 1,154 · cited 3×
“This finding provides supporting evidence of the suspicions of prior studies [44], [62], [127], [145].”
BackgroundResultDangers of Using "Optimal" Cutpoints in the Evaluation of Prognostic Factors
Altman, D. G., Lausen, B., Sauerbrei, W., & Schumacher, M. (1994) · JNCI Journal of the National Cancer Institute · doi:10.1093/jnci/86.11.829 · Cited by 1,143
Search-based software engineering
Harman, M., & Jones, B. F. (2001) · Information and Software Technology · doi:10.1016/s0950-5849(01)00189-6 · Cited by 914
Clinical Prediction Models: A Practical Approach to Development, Validation and Updating
Mann, C. J. H. (2009) · Kybernetes · doi:10.1108/k.2009.06738fae.002 · Cited by 783
neuralnet: Training of Neural Networks
Günther, F., & Fritsch, S. (2010) · The R Journal · doi:10.32614/rj-2010-006 · Cited by 757
Predicting Defects for Eclipse
Zimmermann, T., Premraj, R., & Zeller, A. (2007) · Third International Workshop on Predictor Models in Software Engineering (PROMISE'07: ICSE Workshops 2007) · doi:10.1109/promise.2007.10 · Cited by 753
A systematic review of software development cost estimation studies
Jørgensen, M., & Shepperd, M. (2007) · Brunel University Research Archive (BURA) (Brunel University London) · doi:10.1109/tse.2007.3 · Cited by 742
The PROMISE Repository of Empirical Software Engineering Data
Boetticher, G. D. (2007) · Medical Entomology and Zoology · Cited by 741
A Systematic Review of Software Development Cost Estimation Studies
Jorgensen, M., & Shepperd, M. (2007) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2007.256943 · Cited by 729
An extensive comparison of bug prediction approaches
D'Ambros, M., Lanza, M., & Robbes, R. (2010) · 2010 7th IEEE Working Conference on Mining Software Repositories (MSR 2010) · doi:10.1109/msr.2010.5463279 · Cited by 637
The Current State and Future of Search Based Software Engineering
Harman, M. (2007) · Future of Software Engineering (FOSE '07) · doi:10.1109/fose.2007.29 · Cited by 631
Evaluating defect prediction approaches: a benchmark and an extensive comparison
D’Ambros, M., Lanza, M., & Robbes, R. (2012) · Empirical Software Engineering · doi:10.1007/s10664-011-9173-9 · Cited by 626
Is cross-validation valid for small-sample microarray classification?
Braga-Neto, U. M., & Dougherty, E. R. (2004) · Bioinformatics · doi:10.1093/bioinformatics/btg419 · Cited by 597
An Empirical Comparison of Model Validation Techniques for Defect Prediction Models
Tantithamthavorn, C., McIntosh, S., Hassan, A. E., & Matsumoto, K. (2017) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2016.2584050 · Cited by 574 · cited 14×
“Our recent work shows that defect prediction models that are trained using datasets with low EPV values are especially susceptible to unstable results [141].”
BackgroundMethodInfluentialSelf-citationUsing Class Imbalance Learning for Software Defect Prediction
Wang, S., & Yao, X. (2013) · IEEE Transactions on Reliability · doi:10.1109/tr.2013.2259203 · Cited by 571
Data Quality: Some Comments on the NASA Software Defect Datasets
Shepperd, M., Song, Q., Sun, Z., & Mair, C. (2013) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2013.11 · Cited by 570
Towards identifying software project clusters with regard to defect prediction
Jureczko, M., & Madeyski, L. (2010) · Proceedings of the 6th International Conference on Predictive Models in Software Engineering · doi:10.1145/1868328.1868342 · Cited by 546
The meanings and measurement of clinical significance.
Kazdin, A. E. (1999) · Journal of Consulting and Clinical Psychology · doi:10.1037//0022-006x.67.3.332 · Cited by 522
Transfer learning for cross-company software defect prediction
Ma, Y., Luo, G., Zeng, X., & Chen, A. (2012) · Information and Software Technology · doi:10.1016/j.infsof.2011.09.007 · Cited by 475
rms: Regression Modeling Strategies
Harrell Jr, F. E. (2009) · CRAN: Contributed Packages · doi:10.32614/cran.package.rms · Cited by 458
Arisholm, E., Briand, L. C., & Johannessen, E. B. (2010) · Journal of Systems and Software · doi:10.1016/j.jss.2009.06.055 · Cited by 455
Use of Brier score to assess binary predictions
Rufibach, K. (2010) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2009.11.009 · Cited by 454
Statistics notes: Transformations, means, and confidence intervals: Fig 1
Bland, J. M., & Altman, D. G. (1996) · BMJ · doi:10.1136/bmj.312.7038.1079 · Cited by 423
ReLink: recovering links between bugs and changes
Wu, R., Zhang, H., Kim, S., & Cheung, S.-C. (2011) · Proceedings of the 19th ACM SIGSOFT symposium and the 13th European conference on Foundations of software engineering · doi:10.1145/2025113.2025120 · Cited by 412
Parameter tuning or default values? An empirical investigation in search-based software engineering
Arcuri, A., & Fraser, G. (2013) · Empirical Software Engineering · doi:10.1007/s10664-013-9249-9 · Cited by 384
Software defect prediction using ensemble learning on selected features
Laradji, I. H., Alshayeb, M., & Ghouti, L. (2015) · Information and Software Technology · doi:10.1016/j.infsof.2014.07.005 · Cited by 380
A General Software Defect-Proneness Prediction Framework
Song, Q., Jia, Z., Shepperd, M., Ying, S., & Liu, J. (2011) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2010.90 · Cited by 374
Revisiting the Impact of Classification Techniques on the Performance of Defect Prediction Models
Ghotra, B., McIntosh, S., & Hassan, A. E. (2015) · 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering · doi:10.1109/icse.2015.91 · Cited by 364
Self-citationResearcher Bias: The Use of Machine Learning in Software Defect Prediction
Shepperd, M., Bowes, D., & Hall, T. (2014) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2014.2322358 · Cited by 359
randomForest: Breiman and Cutlers Random Forests for Classification and Regression
Breiman, L., Cutler, A., Liaw, A., & Wiener, M. (2002) · CRAN: Contributed Packages · doi:10.32614/cran.package.randomforest · Cited by 358
McIntosh, S., Kamei, Y., Adams, B., & Hassan, A. E. (2014) · Proceedings of the 11th Working Conference on Mining Software Repositories · doi:10.1145/2597073.2597076 · Cited by 348
Self-citationFair and balanced?: bias in bug-fix datasets
Bird, C., Bachmann, A., Aune, E., Duffy, J., Bernstein, A., Filkov, V., & Devanbu, P. (2009) · Proceedings of the 7th joint meeting of the European software engineering conference and the ACM SIGSOFT symposium on The foundations of software engineering · doi:10.1145/1595696.1595716 · Cited by 335
An empirical study of the impact of modern code review practices on software quality
McIntosh, S., Kamei, Y., Adams, B., & Hassan, A. E. (2016) · Empirical Software Engineering · doi:10.1007/s10664-015-9381-9 · Cited by 332
Self-citationAutomated parameter optimization of classification techniques for defect prediction models
Tantithamthavorn, C., McIntosh, S., Hassan, A. E., & Matsumoto, K. (2016) · Proceedings of the 38th International Conference on Software Engineering · doi:10.1145/2884781.2884857 · Cited by 331
Self-citationKnowledge transfer via multiple model local structure mapping
Gao, J., Fan, W., Jiang, J., & Han, J. (2008) · Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1401890.1401928 · Cited by 330
Dealing with noise in defect prediction
Kim, S., Zhang, H., Wu, R., & Gong, L. (2011) · Proceedings of the 33rd International Conference on Software Engineering · doi:10.1145/1985793.1985859 · Cited by 330
Tantithamthavorn, C., Hassan, A. E., & Matsumoto, K. (2020) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2018.2876537 · Cited by 327
Self-citationA controlled random search procedure for global optimisation
Price, W. L. (1977) · The Computer Journal · doi:10.1093/comjnl/20.4.367 · Cited by 323
Parallel recombinative simulated annealing: A genetic algorithm
Mahfoud, S. W., & Goldberg, D. E. (1995) · Parallel Computing · doi:10.1016/0167-8191(94)00071-h · Cited by 320
It's not a bug, it's a feature: How misclassification impacts bug prediction
Herzig, K., Just, S., & Zeller, A. (2013) · 2013 35th International Conference on Software Engineering (ICSE) · doi:10.1109/icse.2013.6606585 · Cited by 294
Robust Prediction of Fault-Proneness by Random Forests
Guo, L., Ma, Y., Cukic, B., & Singh, H. (2005) · 15th International Symposium on Software Reliability Engineering · doi:10.1109/issre.2004.35 · Cited by 271
Techniques for evaluating fault prediction models
Jiang, Y., Cukic, B., & Ma, Y. (2008) · Empirical Software Engineering · doi:10.1007/s10664-008-9079-3 · Cited by 267
Reliability and validity in comparative studies of software prediction models
Myrtveit, I., Stensrud, E., & Shepperd, M. (2005) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2005.58 · Cited by 259
Revisiting the Impact of Classification Techniques on the Performance of Defect Prediction Models
Ghotra, B., McIntosh, S., & Hassan, A. E. (2015) · Cited by 256
Self-citationHow, and why, process metrics are better
Rahman, F., & Devanbu, P. (2013) · 2013 35th International Conference on Software Engineering (ICSE) · doi:10.1109/icse.2013.6606589 · Cited by 248
Recalling the "imprecision" of cross-project defect prediction
Rahman, F., Posnett, D., & Devanbu, P. (2012) · Proceedings of the ACM SIGSOFT 20th International Symposium on the Foundations of Software Engineering · doi:10.1145/2393596.2393669 · Cited by 246
Search Based Software Engineering: Techniques, Taxonomy, Tutorial
Harman, M., McMinn, P., de Souza, J. T., & Yoo, S. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-25231-0_1 · Cited by 245
Personalized defect prediction
Jiang, T., Tan, L., & Kim, S. (2013) · 2013 28th IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase.2013.6693087 · Cited by 240
Ranking and Clustering Software Cost Estimation Models through a Multiple Comparisons Algorithm
Mittas, N., & Angelis, L. (2013) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2012.45 · Cited by 234
Online Defect Prediction for Imbalanced Data
Tan, M., Tan, L., Dara, S., & Mayeux, C. (2015) · 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering · doi:10.1109/icse.2015.139 · Cited by 229
Easy over hard: a case study on deep learning
Fu, W., & Menzies, T. (2017) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3106237.3106256 · Cited by 228
Zhou, Z.-H. (2009) · Encyclopedia of Biometrics · doi:10.1007/978-0-387-73003-5_293 · Cited by 228
Panichella, A., Dit, B., Oliveto, R., Di Penta, M., Poshynanyk, D., & De Lucia, A. (2013) · 2013 35th International Conference on Software Engineering (ICSE) · doi:10.1109/icse.2013.6606598 · Cited by 216
Nearest prototype classification: clustering, genetic algorithms, or random search?
Kuncheva, L. I., & Bezdek, J. C. (1998) · IEEE Transactions on Systems, Man and Cybernetics, Part C (Applications and Reviews) · doi:10.1109/5326.661099 · Cited by 214
Investigating code review quality: Do people and participation matter?
Kononenko, O., Baysal, O., Guerrouj, L., Cao, Y., & Godfrey, M. W. (2015) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsm.2015.7332457 · Cited by 210
Tuning for software analytics: Is it really necessary?
Fu, W., Menzies, T., & Shen, X. (2016) · Information and Software Technology · doi:10.1016/j.infsof.2016.04.017 · Cited by 210
Revisiting common bug prediction findings using effort-aware models
Kamei, Y., Matsumoto, S., Monden, A., Matsumoto, K.-I., Adams, B., & Hassan, A. E. (2010) · 2010 IEEE International Conference on Software Maintenance · doi:10.1109/icsm.2010.5609530 · Cited by 207
Self-citationPanichella, A., Dit, B., Oliveto, R., Penta, M. D., Poshynanyk, D., & Lucia, A. D. (2013) · Archivio Istituzionale della Ricerca - Università degli Studi del Molise · Cited by 206
Exploiting the Essential Assumptions of Analogy-Based Effort Estimation
Kocaguneli, E., Menzies, T., Bener, A., & Keung, J. W. (2012) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2011.27 · Cited by 200
The missing links: bugs and bug-fix commits
Bachmann, A., Bird, C., Rahman, F., Devanbu, P., & Bernstein, A. (2010) · Proceedings of the eighteenth ACM SIGSOFT international symposium on Foundations of software engineering · doi:10.1145/1882291.1882308 · Cited by 190
Predicting fault-prone components in a java legacy system
Arisholm, E., & Briand, L. C. (2006) · Proceedings of the 2006 ACM/IEEE international symposium on Empirical software engineering · doi:10.1145/1159733.1159738 · Cited by 187
How, and Why, Process Metrics are better
Rahman, F., & Dévanbu, P. (2013) · Cited by 184
NEURAL NETWORKS IN DATA MINING
Singh, Y., & Chauhan, A. S. (2009) · Cited by 182
An empirical study of just-in-time defect prediction using cross-project models
Fukushima, T., Kamei, Y., McIntosh, S., Yamashita, K., & Ubayashi, N. (2014) · Proceedings of the 11th Working Conference on Mining Software Repositories · doi:10.1145/2597073.2597075 · Cited by 180
Self-citationRevisiting the evaluation of defect prediction models
Mende, T., & Koschke, R. (2009) · Proceedings of the 5th International Conference on Predictor Models in Software Engineering · doi:10.1145/1540438.1540448 · Cited by 179
Thongtanunam, P., McIntosh, S., Hassan, A. E., & Iida, H. (2016) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/2884781.2884852 · Cited by 179
Self-citationDo code review practices impact design quality? A case study of the Qt, VTK, and ITK projects
Morales, R., McIntosh, S., & Khomh, F. (2015) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner.2015.7081827 · Cited by 178
Self-citationCross-project defect prediction models: L'Union fait la force
Panichella, A., Oliveto, R., & De Lucia, A. (2014) · 2014 Software Evolution Week - IEEE Conference on Software Maintenance, Reengineering, and Reverse Engineering (CSMR-WCRE) · doi:10.1109/csmr-wcre.2014.6747166 · Cited by 176
An industrial study on the risk of software changes
Shihab, E., Hassan, A. E., Adams, B., & Jiang, Z. M. (2012) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/2393596.2393670 · Cited by 175
Self-citationComparative Assessment of Software Quality Classification Techniques: An Empirical Case Study
Khoshgoftaar, T. M., & Seliya, N. (2004) · Empirical Software Engineering · doi:10.1023/b:emse.0000027781.18360.9b · Cited by 172
Bias, Variance, and Arcing Classifiers
Breiman, L. (1996) · Cited by 171
On Parameter Tuning in Search Based Software Engineering
Arcuri, A., & Fraser, G. (2011) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-23716-4_6 · Cited by 170
Weyuker, E. J., Ostrand, T. J., & Bell, R. M. (2008) · Empirical Software Engineering · doi:10.1007/s10664-008-9082-8 · Cited by 169
The Impact of Mislabelling on the Performance and Interpretation of Defect Prediction Models
Tantithamthavorn, C. K., McIntosh, S., Hassan, A. E., Ihara, A., & Matsumoto, K. (2015) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse.2015.93 · Cited by 169
Self-citationOnline defect prediction for imbalanced data
Tan, M., Tan, L., Dara, S., & Mayeux, C. (2015) · doi:10.5555/2819009.2819026 · Cited by 151
The Impact of Feature Selection on Defect Prediction Performance: An Empirical Comparison
Zhou, X., Liu, J., Yang, Z., An, G., & Jia, X. (2016) · Proceedings/Proceedings - International Symposium on Software Reliability Engineering · doi:10.1109/issre.2016.13 · Cited by 150
An experience report on defect modelling in practice: pitfalls and challenges
Tantithamthavorn, C. K., & Hassan, A. E. (2018) · IEEE/ACM International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP) · doi:10.1145/3183519.3183547 · Cited by 148
Self-citationA Large-Scale Study of the Impact of Feature Selection Techniques on Defect Classification Models
Ghotra, B., McIntosh, S., & Hassan, A. E. (2017) · International Conference on Mining Software Repositories (MSR) · doi:10.1109/msr.2017.18 · Cited by 142
Self-citationComments on "Data Mining Static Code Attributes to Learn Defect Predictors"
Zhang, H., & Zhang, X. (2007) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2007.70706 · Cited by 140
Balancing Privacy and Utility in Cross-Company Defect Prediction
Peters, F., Menzies, T., Gong, L., & Zhang, H. (2013) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2013.6 · Cited by 139
Special issue on repeatable results in software engineering prediction
Menzies, T., & Shepperd, M. (2012) · Empirical Software Engineering · doi:10.1007/s10664-011-9193-5 · Cited by 122
Neural Networks For Data Mining
Zhang, G. P. (2009) · Springer eBooks · doi:10.1007/978-0-387-09823-4_21 · Cited by 121
Sample size vs. bias in defect prediction
Rahman, F., Posnett, D., Herraiz, I., & Dévanbu, P. (2013) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/2491411.2491418 · Cited by 117
Towards building a universal defect prediction model with rank transformed predictors
Zhang, F., Mockus, A., Keivanloo, I., & Zou, Y. (2015) · Empirical Software Engineering · doi:10.1007/s10664-015-9396-2 · Cited by 116
Can data transformation help in the detection of fault-prone modules?
Jiang, Y., Čukić, B., & Menzies, T. (2008) · doi:10.1145/1390817.1390822 · Cited by 104
The Impact of Classifier Configuration and Classifier Combination on Bug Localization
Thomas, S. W., Nagappan, M., Blostein, D., & Hassan, A. E. (2013) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2013.27 · Cited by 102
Self-citationAn investigation of the effect of module size on defect prediction using static measures
Koru, A. G., & Liu, H. (2005) · ACM SIGSOFT Software Engineering Notes · doi:10.1145/1082983.1083172 · Cited by 100
Performance of prognostic models in critically ill cancer patients – a review
den Boer, S. D., de Keizer, N. D., & de Jonge, E. (2005) · Critical Care · doi:10.1186/cc3765 · Cited by 99
Feed-Forward Neural Networks and Multinomial Log-Linear Models
Ripley, B. D. (2015) · Cited by 98
High-impact defects: a study of breakage and surprise defects
Shihab, E., Mockus, A., Kamei, Y., Adams, B., & Hassan, A. E. (2011) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/2025113.2025155 · Cited by 96
Self-citationMair, C., & Shepperd, M. (2005) · International Symposium on Empirical Software Engineering · doi:10.1109/isese.2005.1541858 · Cited by 91
The impact of parameter tuning on software effort estimation using learning machines
Song, L., Minku, L. L., & Yao, X. (2013) · doi:10.1145/2499393.2499394 · Cited by 86
A Genetic Algorithm to Configure Support Vector Machines for Predicting Fault-Prone Components
Martino, S. D., Ferrucci, F., Gravino, C., & Sarro, F. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-21843-9_20 · Cited by 85
The Impact of Using Regression Models to Build Defect Classifiers
Rajbahadur, G. K., Wang, S., Kamei, Y., & Hassan, A. E. (2017) · International Conference on Mining Software Repositories (MSR) · doi:10.1109/msr.2017.4 · Cited by 80
Self-citationImproving trace accuracy through data-driven configuration and composition of tracing features
Lohar, S., Amornborvornwong, S., Zisman, A., & Cleland‐Huang, J. (2013) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/2491411.2491432 · Cited by 79
Global vs. local models for cross-project defect prediction: A replication study
Herbold, S., Trautsch, A., & Grabowski, J. (2016) · Empirical Software Engineering · doi:10.1007/s10664-016-9468-y · Cited by 78
Comments on “Researcher Bias: The Use of Machine Learning in Software Defect Prediction”
Tantithamthavorn, C. K., McIntosh, S., Hassan, A. E., & Matsumoto, K. (2016) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2016.2553030 · Cited by 76
Self-citationLACE2: Better Privacy-Preserving Data Sharing for Cross Project Defect Prediction
Peters, F., Menzies, T., & Layman, L. M. (2015) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse.2015.92 · Cited by 75
The jinx on the NASA software defect data sets
Petrić, J., Bowes, D., Hall, T., Christianson, B., & Baddoo, N. (2016) · International Conference on Evaluation and Assessment in Software Engineering (EASE) · doi:10.1145/2915970.2916007 · Cited by 70
Privacy and utility for defect prediction: Experiments with MORPH
Peters, F., & Menzies, T. (2012) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse.2012.6227194 · Cited by 69
Reducing false alarms in software defect prediction by decision threshold optimization
Tosun, A., & Bener, A. (2009) · Proceedings of the ... ACM-IEEE International Symposium on Empirical Software Engineering and Measurement/International Symposium on Empirical Software Engineering and Measurement · doi:10.1109/esem.2009.5316006 · Cited by 67
A Case Study of Bias in Bug-Fix Datasets
Nguyen, T., Adams, B., & Hassan, A. E. (2010) · Working Conference on Reverse Engineering (WCRE) · doi:10.1109/wcre.2010.37 · Cited by 66
Self-citationJustification of Logarithmic Loss via the Benefit of Side Information
Jiao, J., Courtade, T. A., Venkat, K., & Weissman, T. (2015) · IEEE Transactions on Information Theory · doi:10.1109/tit.2015.2462848 · Cited by 64
Replication of defect prediction studies: problems, pitfalls and recommendations
Mende, T. (2010) · doi:10.1145/1868328.1868336 · Cited by 64
Studying the impact of social interactions on software quality
Bettenburg, N., & Hassan, A. E. (2012) · Empirical Software Engineering · doi:10.1007/s10664-012-9205-0 · Cited by 62
Self-citationWhy is Differential Evolution Better than Grid Search for Tuning Defect Predictors?Preprint
Fu, W., Nair, V., & Menzies, T. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1609.02613 · Cited by 62
Software Defect Prediction Using Regression via Classification
Bibi, S., Tsoumakas, G., Stamelos, I. G., & Vlahvas, I. (2006) · IEEE/ACS International Conference on Computer Systems and Applications (AICCSA) · doi:10.1109/aiccsa.2006.205110 · Cited by 61
C50: C5.0 Decision Trees and Rule-Based Models
Kühn, M., & Quinlan, R. (2012) · doi:10.32614/cran.package.c50 · Cited by 61
Adequate and Precise Evaluation of Quality Models in Software Engineering Studies
Ma, Y., & Čukić, B. (2007) · Predictive Models in Software Engineering · doi:10.1109/promise.2007.1 · Cited by 46
Privacy and utility for defect prediction: experiments with MORPH
Peters, F., & Menzies, T. (2012) · doi:10.5555/2337223.2337246 · Cited by 43
Ensemble of software defect predictors
Tosun, A., Turhan, B., & Bener, A. (2008) · Proceedings of the ... ACM-IEEE International Symposium on Empirical Software Engineering and Measurement/International Symposium on Empirical Software Engineering and Measurement · doi:10.1145/1414004.1414066 · Cited by 40
A Study of Redundant Metrics in Defect Prediction Datasets
Jiarpakdee, J., Tantithamthavorn, C. K., Ihara, A., & Matsumoto, K. (2016) · IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW) · doi:10.1109/issrew.2016.30 · Cited by 37
Self-citationThe Impact of Correlated Metrics on Defect ModelsPreprint
Jiarpakdee, J., Tantithamthavorn, C. K., & Hassan, A. E. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1801.10271 · Cited by 34
Self-citationTantithamthavorn, C. K. (2016) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/2889160.2889256 · Cited by 28
Self-citationEvaluating Defect Prediction Models for a Large Evolving Software System
Mende, T., Koschke, R., & Leszak, M. (2009) · Euromicro Conference on Software Maintenance and Reengineering (CSMR) · doi:10.1109/csmr.2009.55 · Cited by 26
Different Classifiers Find Different Defects Although With Different Level of Consistency
Bowes, D., Hall, T., & Petrić, J. (2015) · doi:10.1145/2810146.2810149 · Cited by 11
Verification of forecasts expressed in terms of probability
BRIER, G. W. (1950) · Monthly Weather Review · doi:10.1175/1520-0493(1950)078%3c0001:vofeit%3e2.0.co;2
VERIFICATION OF FORECASTS EXPRESSED IN TERMS OF PROBABILITY
BRIER, G. W. (1950) · Monthly Weather Review · doi:10.1175/1520-0493(1950)078<0001:vofeit>2.0.co;2
(2014) · doi:10.5555/2627435.2697065
Cited by
Tantithamthavorn, C., Hassan, A. E., & Matsumoto, K. (2020) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2018.2876537 · Cited by 327
Self-citationGenerative Adversarial Networks for face generation: A survey
Kammoun, A., Slama, R., Tabia, H., Ouni, T., & Abid, M. (2022) · ACM Computing Surveys · doi:10.1145/1122445.1122456 · Cited by 311
Azeem, M. I., Palomba, F., Shi, L., & Wang, Q. (2019) · Information and Software Technology · doi:10.1016/j.infsof.2018.12.009 · Cited by 289
An Empirical Study of Model-Agnostic Techniques for Defect Prediction Models
Jiarpakdee, J., Tantithamthavorn, C. K., Dam, H. K., & Grundy, J. (2022) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2020.2982385 · Cited by 148
Self-citationPredicting Defective Lines Using a Model-Agnostic Technique
Wattanakriengkrai, S., Thongtanunam, P., Tantithamthavorn, C., Hata, H., & Matsumoto, K. (2022) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2020.3023177 · Cited by 137
Self-citation
Metadata
- Authors
- Tantithamthavorn, C., McIntosh, S., Hassan, A. E., & Matsumoto, K.
- Venue
- IEEE Transactions on Software Engineering
- Published
- 2019-07-01
- Volume
- 45
- Issue
- 7
- Pages
- 683-711
- ISSN
- 0098-5589 (ISSN-L)
- 1939-3520
- 2326-3881
- OpenAlex
- W2963520355
- Cited by
- 415
- References
- 151
- FWCI
- 63.47
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.