The DOI matches its Crossref record; 38 of the 38 references it cites were found in a registry or an index.
The choice of scaling technique matters for classification performance
Open accessAbstract
Dataset scaling, also known as normalization, is an essential preprocessing step in a machine learning pipeline. It is aimed at adjusting attributes scales in a way that they all vary within the same range. This transformation is known to improve the performance of classification models, but there are several scaling techniques to choose from, and this choice is not generally done carefully. In this paper, we execute a broad experiment comparing the impact of 5 scaling techniques on the performances of 20 classification algorithms among monolithic and ensemble models, applying them to 82 publicly available datasets with varying imbalance ratios. Results show that the choice of scaling technique matters for classification performance, and the performance difference between the best and the worst scaling technique is relevant and statistically significant in most cases. They also indicate that choosing an inadequate technique can be more detrimental to classification performance than not scaling the data at all. We also show how the performance variation of an ensemble model, considering different scaling techniques, tends to be dictated by that of its base model. Finally, we discuss the relationship between a model's sensitivity to the choice of scaling technique and its performance and provide insights into its applicability on different model deployment scenarios. Full results and source code for the experiments in this paper are available in a GitHub repository.\footnote{https://github.com/amorimlb/scaling\_matters}
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538
Scikit-learn: Machine Learning in Python
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Müller, A., Nothman, J., Louppe, G., et al. (2012) · arXiv · doi:10.48550/arxiv.1201.0490 · Cited by 71,016
Cortes, C., & Vapnik, V. (1995) · Machine Learning · doi:10.1007/bf00994018 · Cited by 59,421 · cited 1×
“This paper includes the following instance-based algorithms in the experiments: the k -Nearest Neighbors (KNN), GLVQ [13] and SVM [14].”
XGBoost: A Scalable Tree Boosting System
Chen, T., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939785 · Cited by 54,569
SciPy 1.0: fundamental algorithms for scientific computing in Python
Virtanen, P., Gommers, R., Oliphant, T. E., Haberland, M., Reddy, T., Cournapeau, D., Burovski, E., Peterson, P., Weckesser, W., Bright, J., et al. (2020) · Nature Methods · doi:10.1038/s41592-019-0686-2 · Cited by 38,430
Greedy function approximation: A gradient boosting machine
Friedman, J. H. (2001) · The Annals of Statistics · doi:10.1214/aos/1013203451 · Cited by 30,731
UCI Machine Learning Repository
Asuncion, A. (2007) · Medical Entomology and Zoology · Cited by 25,410
A decision-theoretic generalization of on-line learning and an application to boosting
Freund, Y., & Schapire, R. E. (1997) · Journal of Computer and System Sciences · doi:10.1006/jcss.1997.1504 · Cited by 20,466
Nearest neighbor pattern classification
Cover, T., & Hart, P. (1967) · IEEE Transactions on Information Theory · doi:10.1109/tit.1967.1053964 · Cited by 16,936
Breiman, L. (1996) · Machine Learning · doi:10.1007/bf00058655 · Cited by 16,325
Classification And Regression Trees
Breiman, L., Friedman, J. H., Olshen, R. A., & Stone, C. J. (2017) · doi:10.1201/9781315139470 · Cited by 11,417
SciPy 1.0: fundamental algorithms for scientific computing in Python
Virtanen, P., Gommers, R., Oliphant, T. E., Haberland, M., Reddy, T., Cournapeau, D., Burovski, E., Peterson, P., Weckesser, W., Bright, J., et al. (2019) · Apollo - University of Cambridge Repository · doi:10.17863/cam.64134 · Cited by 10,460
XGBoost: A Scalable Tree Boosting System
Chen, T., & Guestrin, C. (2016) · arXiv · doi:10.48550/arxiv.1603.02754 · Cited by 7,104
Alcalá‐Fdez, J., Fernández, A., Luengo, J., Derrac, J., García, S., Sánchez, L., & Herrera, F. (2011) · Repositorio de la Universidad de Oviedo (RUO) · Cited by 2,729
Ensemble Methods: Foundations and Algorithms
Zhou, Z.-H. (2012) · doi:10.1201/b12207 · Cited by 2,534
Ensemble Methods: Foundations and Algorithms
Zhou, Z. (2012) · Cited by 2,051
Encyclopedia of Database Systems
(2009) · Springer eBooks · doi:10.1007/978-0-387-39940-9 · Cited by 1,739
Investigating the impact of data normalization on classification performance
Singh, D., & Singh, B. (2020) · Applied Soft Computing · doi:10.1016/j.asoc.2019.105524 · Cited by 1,726
The Optimality of Naive Bayes.
Zhang, H. (2004) · Cited by 1,550
GAUSSIAN PROCESSES FOR MACHINE LEARNING
SEEGER, M. (2004) · International Journal of Neural Systems · doi:10.1142/s0129065704001899 · Cited by 1,488
Combination of multiple classifiers using local accuracy estimates
Woods, K., Kegelmeyer, W. P., & Bowyer, K. (1997) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.588027 · Cited by 952
A theoretical study on six classifier fusion strategies
Kuncheva, L. I. (2002) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.982906 · Cited by 720
Generalized Learning Vector Quantization
Sato, A., & Yamada, K. (1995) · Cited by 475
From dynamic classifier selection to dynamic ensemble selection
Ko, A. H. R., Sabourin, R., & Britto, Jr. (2008) · Pattern Recognition · doi:10.1016/j.patcog.2007.10.015 · Cited by 442
Dynamic classifier selection: Recent advances and perspectives
Cruz, R. M. O., Sabourin, R., & Cavalcanti, G. D. C. (2018) · Information Fusion · doi:10.1016/j.inffus.2017.09.010 · Cited by 440
Self-citationNeural Networks and Deep Learning: A Textbook
Aggarwal, C. C. (2018) · doi:10.1007/978-3-319-94463-0 · Cited by 417
Raju, V. N. G., Lakshmi, K. P., Jain, V. M., Kalidindi, A., & Padma, V (2020) · 2020 Third International Conference on Smart Systems and Inventive Technology (ICSSIT) · doi:10.1109/icssit48917.2020.9214160 · Cited by 345
Dynamic selection of classifiers—A comprehensive review
Britto, A. S., Sabourin, R., & Oliveira, L. E. S. (2014) · Pattern Recognition · doi:10.1016/j.patcog.2014.05.003 · Cited by 331
META-DES: A dynamic ensemble selection framework using meta-learning
Cruz, R. M. O., Sabourin, R., Cavalcanti, G. D. C., & Ing Ren, T. (2015) · Pattern Recognition · doi:10.1016/j.patcog.2014.12.003 · Cited by 250
Self-citationLeveraging the Feature Distribution in Transfer-Based Few-Shot Learning
Hu, Y., Gripon, V., & Pateux, S. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-86340-1_39 · Cited by 224
Review of Classifier Combination Methods
Tulyakov, S., Jaeger, S., Govindaraju, V., & Doermann, D. (2008) · Studies in Computational Intelligence · doi:10.1007/978-3-540-76280-5_14 · Cited by 219
Dynamic selection of normalization techniques using data complexity measures
Jain, S., Shukla, S., & Wadhvani, R. (2018) · Expert Systems with Applications · doi:10.1016/j.eswa.2018.04.008 · Cited by 215
Keun, H. C., Ebbels, T. M. D., Antti, H., Bollard, M. E., Beckonert, O., Holmes, E., Lindon, J. C., & Nicholson, J. K. (2003) · Analytica Chimica Acta · doi:10.1016/s0003-2670(03)00094-1 · Cited by 192
Machine Learning in Document Analysis and Recognition
Marinai, S., & Fujisawa, H. (2007) · Studies in computational intelligence · doi:10.1007/978-3-540-76280-5 · Cited by 95
Artificial Neural Networks and Machine Learning – ICANN 2021
(2021) · Lecture notes in computer science · doi:10.1007/978-3-030-86362-3 · Cited by 46
Hutchison, D., Mattern, F., Takeo Kanade, A., Kleinberg, J. M., Nierstrasz, O., Pandu Rangan, C., Vardi, M. Y., Sudan, M., Mitchell, J. C., Tygar, D., et al. (2009) · Lecture notes in computer science · doi:10.1007/978-3-642-02326-2 · Cited by 33
Online local pool generation for dynamic classifier selection
Souza, M. A., Cavalcanti, G. D. C., Cruz, R. M. O., & Sabourin, R. (2018) · Pattern Recognition · doi:10.1016/j.patcog.2018.08.004 · Cited by 32
Self-citationDzierżak, R. (2019) · Informatyka Automatyka Pomiary w Gospodarce i Ochronie Środowiska · doi:10.35784/iapgos.62 · Cited by 18
Cited by
Kırbaş, İ., & Çifci, A. (2025) · Balkan Journal of Electrical and Computer Engineering · doi:10.17694/bajece.1577929 · Cited by 5
Metadata
- Authors
- de Amorim, L. B. V., Cavalcanti, G. D. C., & Cruz, R. M. O.
- Venue
- Applied Soft Computing
- Publisher
- Elsevier BV
- Published
- 2023-01
- Volume
- 133
- Pages
- 109924
- ISSN
- 1568-4946 (ISSN-L)
- OpenAlex
- W4311543603
- Cited by
- 382
- References
- 38
- FWCI
- 22.86
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.