Detecting code smells using machine learning techniques: Are we there yet?
Open accessThe DOI matches its Crossref record; 82 of the 82 references it cites were found in a registry or an index.
Abstract
Code smells are symptoms of poor design and implementation choices weighing heavily on the quality of produced source code. During the last decades several code smell detection tools have been proposed. However, the literature shows that the results of these tools can be subjective and are intrinsically tied to the nature and approach of the detection. In a recent work the use of Machine-Learning (ML) techniques for code smell detection has been proposed, possibly solving the issue of tool subjectivity giving to a learner the ability to discern between smelly and non-smelly source code elements. While this work opened a new perspective for code smell detection, it only considered the case where instances affected by a single type smell are contained in each dataset used to train and test the machine learners. In this work we replicate the study with a different dataset configuration containing instances of more than one type of smell. The results reveal that with this configuration the machine learning techniques reveal critical limitations in the state of the art which deserve further research.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538
LIBSVM: A library for support vector machines
Chang, C.-C., & Lin, C.-J. (2011) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/1961189.1961199 · Cited by 40,918
SMOTE synthetic minority over-sampling technique
Chawla, N. V., Bowyer, K. W., Hall, L. O., & Kegelmeyer, W. P. (2002) · Journal of Artificial Intelligence Research · doi:10.1613/jair.953 · Cited by 32,865
(1992) · Cited by 23,431
Quinlan, J. R. (1986) · Machine Learning · doi:10.1007/bf00116251 · Cited by 21,439
The WEKA data mining software: an update
Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., & Witten, I. H. (2009) · ACM SIGKDD Explorations Newsletter · doi:10.1145/1656274.1656278 · Cited by 17,750 · cited 3×
“As for J48, the three types of pruning techniques available in WEKA [67] were used, SMO was based on two kernels (e.g., POLYNOMIAL and RBF), while for LIBSVM eight different configurations, using C-SVC and V-SVC, were used.”
MethodBaeza-Yates, R. A., & Ribeiro-Neto, B. A. (1999) · Cited by 11,505
A Caution Regarding Rules of Thumb for Variance Inflation Factors
O’brien, R. M. (2007) · Quality & Quantity · doi:10.1007/s11135-006-9018-6 · Cited by 11,217
Cross validatory choice and assessment of statistical predictions
Stone, M. (1974) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1974.tb00994.x · Cited by 10,881
Libsvm : A library for support vector machines
Ferrari, V. (2008) · Medical Entomology and Zoology · Cited by 10,122
(2012) · Cited by 8,009
Refactoring: Improving the Design of Existing Code
Fowler, M. (2002) · Lecture Notes in Computer Science · doi:10.1007/3-540-45672-4_31 · Cited by 5,591
A Practical Guide to Support Vector Classication
Hsu, C., Chang, C.-C., & Lin, C. (2008) · Cited by 5,083
Cohen, W. W. (1995) · Machine Learning Proceedings 1995 · doi:10.1016/b978-1-55860-377-6.50023-2 · Cited by 3,787
Estimating Continuous Distributions in Bayesian Classifiers
John, G. H., & Langley, P. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1302.4964 · Cited by 2,885
(1998) · Cited by 2,679
Rokach, L. (2010) · Artificial Intelligence Review · doi:10.1007/s10462-009-9124-7 · Cited by 2,657 · cited 1×
“, a method that iteratively uses a set of models built in previous iterations to manipulate the training set and make it more suitable for the classification problem [69], leading to 32 different variants.”
MethodRefactoring improving the design of existing code
Saca, M. A. (2017) · 2017 IEEE 37th Central America and Panama Convention (CONCAPAN XXXVII) · doi:10.1109/concapan.2017.8278488 · Cited by 2,054
Introduction to Machine Learning
Yu, F. R., & He, Y. (2019) · SpringerBriefs in Electrical and Computer Engineering · doi:10.1007/978-3-030-10546-4_1 · Cited by 1,594
Programs, life cycles, and laws of software evolution
Lehman, M. M. (1980) · Proceedings of the IEEE · doi:10.1109/proc.1980.11805 · Cited by 1,181
Practical Nonparametric Statistics.
Sen, P. K., & Conover, W. J. (1972) · Journal of the American Statistical Association · doi:10.2307/2284744 · Cited by 993
DECOR: A Method for the Specification and Detection of Code and Design Smells
Moha, N., Gueheneuc, Y.-G., Duchien, L., & Le Meur, A.-F. (2010) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2009.50 · Cited by 803
The WyCash portfolio management system
Cunningham, W. (1992) · Addendum to the proceedings on Object-oriented programming systems, languages, and applications (Addendum) - OOPSLA '92 · doi:10.1145/157709.157715 · Cited by 742
Technical Debt: From Metaphor to Theory and Practice
Kruchten, P., Nord, R. L., & Ozkaya, I. (2012) · IEEE Software · doi:10.1109/ms.2012.167 · Cited by 735
Effect sizes for research: A broad practical approach.
Grissom, R. J., & Kim, J. J. (2005) · Cited by 732
Practical Nonparametric Statistics, 3rd Ed.
Borror, C. M. (2001) · Journal of Quality Technology · doi:10.1080/00224065.2001.11980074 · Cited by 583
An Empirical Comparison of Model Validation Techniques for Defect Prediction Models
Tantithamthavorn, C., McIntosh, S., Hassan, A. E., & Matsumoto, K. (2017) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2016.2584050 · Cited by 574
Deep learning code fragments for code clone detection
White, M., Tufano, M., Vendome, C., & Poshyvanyk, D. (2016) · Proceedings of the 31st IEEE/ACM International Conference on Automated Software Engineering · doi:10.1145/2970276.2970326 · Cited by 558
Detection strategies: metrics-based rules for detecting design flaws
Marinescu, R. (2004) · 20th IEEE International Conference on Software Maintenance, 2004. Proceedings. · doi:10.1109/icsm.2004.1357820 · Cited by 551
The WyCash portfolio management system
Cunningham, W. (1993) · ACM SIGPLAN OOPS Messenger · doi:10.1145/157710.157715 · Cited by 545
Object-Oriented Metrics in Practice
(2006) · Springer eBooks · doi:10.1007/3-540-39538-5 · Cited by 527
Introduction to Machine Learning
Baştanlar, Y., & Özuysal, M. (2014) · Methods in Molecular Biology · doi:10.1007/978-1-62703-748-8_7 · Cited by 505
Comparing and experimenting machine learning techniques for code smell detection
Arcelli Fontana, F., Mäntylä, M. V., Zanoni, M., & Marino, A. (2016) · Empirical Software Engineering · doi:10.1007/s10664-015-9378-4 · Cited by 450
Managing technical debt in software-reliant systems
Brown, N., Cai, Y., Guo, Y., Kazman, R., Kim, M., Kruchten, P., Lim, E., MacCormack, A., Nord, R., Ozkaya, I., et al. (2010) · Proceedings of the FSE/SDP workshop on Future of software engineering research · doi:10.1145/1882362.1882373 · Cited by 432
The Qualitas Corpus: A Curated Collection of Java Code for Empirical Studies
Tempero, E., Anslow, C., Dietrich, J., Han, T., Li, J., Lumpe, M., Melton, H., & Noble, J. (2010) · 2010 Asia Pacific Software Engineering Conference · doi:10.1109/apsec.2010.46 · Cited by 407
An exploratory study of the impact of antipatterns on class change- and fault-proneness
Khomh, F., Penta, M. D., Guéhéneuc, Y.-G., & Antoniol, G. (2012) · Empirical Software Engineering · doi:10.1007/s10664-011-9171-y · Cited by 395
Identification of Move Method Refactoring Opportunities
Tsantalis, N., & Chatzigeorgiou, A. (2009) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2009.1 · Cited by 339
Quantifying the Effect of Code Smells on Maintenance Effort
Sjoberg, D. I. K., Yamashita, A., Anda, B. C. D., Mockus, A., & Dyba, T. (2013) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2012.89 · Cited by 316
The role of replications in Empirical Software Engineering
Shull, F. J., Carver, J. C., Vegas, S., & Juristo, N. (2008) · Empirical Software Engineering · doi:10.1007/s10664-008-9060-1 · Cited by 315
Abbes, M., Khomh, F., Gueheneuc, Y.-G., & Antoniol, G. (2011) · 2011 15th European Conference on Software Maintenance and Reengineering · doi:10.1109/csmr.2011.24 · Cited by 309
Mining Version Histories for Detecting Code Smells
Palomba, F., Bavota, G., Penta, M. D., Oliveto, R., Poshyvanyk, D., & De Lucia, A. (2015) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2014.2372760 · Cited by 259
Self-citationDo They Really Smell Bad? A Study on Developers' Perception of Bad Code Smells
Palomba, F., Bavota, G., Di Penta, M., Oliveto, R., & De Lucia, A. (2014) · 2014 IEEE International Conference on Software Maintenance and Evolution · doi:10.1109/icsme.2014.32 · Cited by 251
Self-citationThe evolution and impact of code smells: A case study of two open source systems
Olbrich, S., Cruzes, D. S., Basili, V., & Zazworka, N. (2009) · 2009 3rd International Symposium on Empirical Software Engineering and Measurement · doi:10.1109/esem.2009.5314231 · Cited by 245
When and Why Your Code Starts to Smell Bad (and Whether the Smells Go Away)
Tufano, M., Palomba, F., Bavota, G., Oliveto, R., Penta, M. D., De Lucia, A., & Poshyvanyk, D. (2017) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2017.2653105 · Cited by 236
Self-citationAn experimental investigation on the innate relationship between quality and refactoring
Bavota, G., De Lucia, A., Di Penta, M., Oliveto, R., & Palomba, F. (2015) · Journal of Systems and Software · doi:10.1016/j.jss.2015.05.024 · Cited by 232
Self-citationExploring the impact of inter-smell relations on software maintainability: An empirical study
Yamashita, A. F., & Moonen, L. (2013) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse.2013.6606614 · Cited by 229
A Bayesian Approach for the Detection of Code and Design Smells
Khomh, F., Vaucher, S., Guéhéneuc, Y.-G., & Sahraoui, H. (2009) · 2009 Ninth International Conference on Quality Software · doi:10.1109/qsic.2009.47 · Cited by 224
Do code smells reflect important maintainability aspects?
Yamashita, A., & Moonen, L. (2012) · 2012 28th IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsm.2012.6405287 · Cited by 222
Code Bad Smells: a review of current knowledge
Zhang, M., Hall, T., & Baddoo, N. (2011) · Journal of Software Maintenance and Evolution: Research and Practice · doi:10.1002/smr.521 · Cited by 213
Software defect prediction: do different classifiers find the same defects?
Bowes, D., Hall, T., & Petrić, J. (2018) · Software Quality Journal · doi:10.1007/s11219-016-9353-3 · Cited by 207
Automatic detection of bad smells in code: An experimental assessment.
Arcelli Fontana, F., Braione, P., & Zanoni, M. (2012) · The Journal of Object Technology · doi:10.5381/jot.2012.11.2.a5 · Cited by 203
A review-based comparative study of bad smell detection tools
Fernandes, E., Oliveira, J., Vale, G., Paiva, T., & Figueiredo, E. (2016) · Proceedings of the 20th International Conference on Evaluation and Assessment in Software Engineering · doi:10.1145/2915970.2915984 · Cited by 192
Code smell severity classification using machine learning techniques
Arcelli Fontana, F., & Zanoni, M. (2017) · Knowledge-Based Systems · doi:10.1016/j.knosys.2017.04.014 · Cited by 180
BDTEX: A GQM-based Bayesian approach for the detection of antipatterns
Khomh, F., Vaucher, S., Guéhéneuc, Y., & Sahraoui, H. A. (2010) · Journal of Systems and Software · doi:10.1016/j.jss.2010.11.921 · Cited by 175
Code Smell Detection: Towards a Machine Learning-Based Approach
Fontana, F. A., Zanoni, M., Marino, A., & Mantyla, M. V. (2013) · 2013 IEEE International Conference on Software Maintenance · doi:10.1109/icsm.2013.56 · Cited by 174
Subjective evaluation of software evolvability using code smells: An empirical study
Mäntylä, M. V., & Lassenius, C. (2007) · Empirical Software Engineering · doi:10.1007/s10664-006-9002-8 · Cited by 171 · cited 1×
“In the first place, code smells detected by existing approaches can be subjectively perceived and interpreted by developers [30], [31].”
BackgroundOn negative results when using sentiment analysis tools for software engineering research
Jongeling, R., Sarkar, P., Datta, S., & Serebrenik, A. (2017) · Empirical Software Engineering · doi:10.1007/s10664-016-9493-x · Cited by 171
Self-citationMann, C. J. H. (2007) · Kybernetes · doi:10.1108/k.2007.06736eae.001 · Cited by 164
An empirical investigation into the nature of test smells
Tufano, M., Palomba, F., Bavota, G., Di Penta, M., Oliveto, R., De Lucia, A., & Poshyvanyk, D. (2016) · Proceedings of the 31st IEEE/ACM International Conference on Automated Software Engineering · doi:10.1145/2970276.2970340 · Cited by 160
Self-citationTowards logistic regression models for predicting fault-prone code across software projects
Cruz, A. E. C., & Ochimizu, K. (2009) · Proceedings of the ... ACM-IEEE International Symposium on Empirical Software Engineering and Measurement/International Symposium on Empirical Software Engineering and Measurement · doi:10.1109/esem.2009.5316002 · Cited by 149
Investigating the Evolution of Bad Smells in Object-Oriented Code
Chatzigeorgiou, A., & Manakos, A. (2010) · International Conference on the Quality of Information and Communications Technology (QUATIC) · doi:10.1109/quatic.2010.16 · Cited by 148
A Cooperative Parallel Search-Based Software Engineering Approach for Code-Smells Detection
Kessentini, W., Kessentini, M., Sahraoui, H. A., Bechikh, S., & Ouni, A. (2014) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2014.2331057 · Cited by 134
On the diffusion of test smells in automatically generated test code: an empirical study
Palomba, F., Nucci, D. D., Panichella, A., Oliveto, R., & Lucia, A. D. (2016) · IEEE/ACM International Workshop on Search-Based Software Testing (SBST) · doi:10.1145/2897010.2897016 · Cited by 129
Self-citationReplication of empirical studies in software engineering research: a systematic mapping study
da Silva, F. Q. B., Suassuna, M., França, A. C. C., Grubb, A. M., Gouveia, T. B., Monteiro, C. V. F., & dos Santos, I. E. (2012) · Empirical Software Engineering · doi:10.1007/s10664-012-9227-7 · Cited by 126
SMURF: A SVM-based Incremental Anti-pattern Detection Approach
Maïga, A., Ali, N., Bhattacharya, N., Sabané, A., Guéhéneuc, Y., & Aı̈meur, E. (2012) · Working Conference on Reverse Engineering (WCRE) · doi:10.1109/wcre.2012.56 · Cited by 126
Toward a Smell-Aware Bug Prediction Model
Palomba, F., Zanoni, M., Fontana, F. A., Lucia, A. D., & Oliveto, R. (2017) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2017.2770122 · Cited by 124
Self-citationAn Exploratory Study on the Relationship between Changes and Refactoring
Palomba, F., Zaidman, A., Oliveto, R., & Lucia, A. D. (2017) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1109/icpc.2017.38 · Cited by 122
Self-citationSupport vector machines for anti-pattern detection
Maïga, A., Ali, N., Bhattacharya, N., Sabané, A., Guéhéneuc, Y., Antoniol, G., & Aı̈meur, E. (2012) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/2351676.2351723 · Cited by 116
How developers perceive smells in source code: A replicated study
Taibi, D., Janes, A., & Lenarduzzi, V. (2017) · Information and Software Technology · doi:10.1016/j.infsof.2017.08.008 · Cited by 112
Understanding the longevity of code smells: preliminary results of an explanatory survey
Arcoverde, R., Garcia, A., & Figueiredo, E. (2011) · Workshop on Refactoring Tools (WRT) · doi:10.1145/1984732.1984740 · Cited by 102
Experience report: Evaluating the effectiveness of decision trees for detecting code smells
Amorim, L., Costa, E. D. B., Antunes, N., Fonseca, B., & Ribeiro, M. (2015) · Proceedings/Proceedings - International Symposium on Software Reliability Engineering · doi:10.1109/issre.2015.7381819 · Cited by 85
Automatic test case generation: what if test code quality matters?
Palomba, F., Panichella, A., Zaidman, A., Oliveto, R., & Lucia, A. D. (2016) · International Symposium on Software Testing and Analysis · doi:10.1145/2931037.2931057 · Cited by 79
Self-citationDynamic Selection of Classifiers in Bug Prediction: An Adaptive Method
Nucci, D. D., Palomba, F., Oliveto, R., & Lucia, A. D. (2017) · IEEE Transactions on Emerging Topics in Computational Intelligence · doi:10.1109/tetci.2017.2699224 · Cited by 78
Self-citationThe Scent of a Smell: An Extensive Comparison Between Textual and Structural Smells
Palomba, F., Panichella, A., Zaidman, A., Oliveto, R., & Lucia, A. D. (2017) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2017.2752171 · Cited by 78
Self-citationCan I clone this piece of code here?
Wang, X., Dang, Y., Zhang, L., Zhang, D., Lan, E., & Mei, H. (2012) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/2351676.2351701 · Cited by 71
Itkonen, J., & Mäntylä, M. (2013) · Empirical Software Engineering · doi:10.1007/s10664-013-9266-8 · Cited by 63
The scent of a smell: an extensive comparison between textual and structural smells
Palomba, F., Panichella, A., Zaidman, A., Oliveto, R., & Lucia, A. D. (2018) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3180155.3182530 · Cited by 58
Self-citationPalomba, F., Lucia, A. D., Bavota, G., & Oliveto, R. (2014) · Advances in computers · doi:10.1016/b978-0-12-800160-8.00004-8 · Cited by 52
Self-citationTechnical Debt: Showing the Way for Better Transfer of Empirical Results
Shull, F. J., Falessi, D., Seaman, C. B., Diep, M. M., & Layman, L. M. (2013) · Springer eBooks · doi:10.1007/978-3-642-37395-4_12 · Cited by 50
How do Scratch Programmers Name Variables and Procedures?
Swidan, A., Serebrenik, A., & Hermans, F. (2017) · IEEE International Working Conference on Source Code Analysis and Manipulation (SCAM) · doi:10.1109/scam.2017.12 · Cited by 38
Self-citationDeveloping Fault-Prediction Models: What the Research Can Show Industry
Hall, T., Beecham, S., Bowes, D., Gray, D. H., & Counsell, S. (2011) · IEEE Software · doi:10.1109/ms.2011.138 · Cited by 27
(1996)
Cited by
Azeem, M. I., Palomba, F., Shi, L., & Wang, Q. (2019) · Information and Software Technology · doi:10.1016/j.infsof.2018.12.009 · Cited by 289
Self-citationAn Empirical Study of Code Smells in Transformer-based Code Generation Techniques
Siddiq, M. L., Majumder, S. H., Mim, M. R., Jajodia, S., & Santos, J. C. S. (2022) · 2022 IEEE 22nd International Working Conference on Source Code Analysis and Manipulation (SCAM) · doi:10.1109/scam55253.2022.00014 · Cited by 70
“[57] described machine learning-based experiments with a new dataset configuration that includes instances of multiple types of smells.”
BackgroundPredicting Design Impactful Changes in Modern Code Review: A Large-Scale Empirical Study
Uchoa, A., Barbosa, C., Coutinho, D., Oizumi, W., Assuncao, W. K. G., Vergilio, S. R., Pereira, J. A., Oliveira, A., & Garcia, A. (2021) · 2021 IEEE/ACM 18th International Conference on Mining Software Repositories (MSR) · doi:10.1109/msr52588.2021.00059 · Cited by 42
“In fact, ML has been used intensively for smell characterization and detection [61], [62], [69], [70].”
MethodSecurity in DevSecOps: Applying Tools and Machine Learning to Verification and Monitoring Steps
Cankar, M., Petrovic, N., Pita Costa, J., Cernivec, A., Antic, J., Martincic, T., & Stepec, D. (2023) · Companion of the 2023 ACM/SPEC International Conference on Performance Engineering · doi:10.1145/3578245.3584943 · Cited by 24
“The available of-the-shelf solutions – also called checks – can base on the expert knowledge integrated in the tool [6] or rely on machine learning approaches [4].”
Background
Metadata
- Authors
- Di Nucci, D., Palomba, F., Tamburri, D. A., Serebrenik, A., & De Lucia, A.
- Venue
- 2018 IEEE 25th International Conference on Software Analysis, Evolution and Reengineering (SANER)
- Publisher
- IEEE
- Published
- 2018-03
- Pages
- 612-621
- OpenAlex
- W2796404405
- Cited by
- 305
- References
- 82
- FWCI
- 23.72
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.