An Overview of Overfitting and its Solutions
Open accessAbstract
Overfitting is a fundamental issue in supervised machine learning which prevents us from perfectly generalizing the models to well fit observed data on training data, as well as unseen data on testing set. Because of the presence of noise, the limited size of training set, and the complexity of classifiers, overfitting happens. This paper is going to talk about overfitting from the perspectives of causes and solutions. To reduce the effects of overfitting, various strategies are proposed to address to these causes: 1) "early-stopping" strategy is introduced to prevent overfitting by stopping training before the performance stops optimize; 2) "network-reduction" strategy is used to exclude the noises in training set; 3) "data-expansion" strategy is proposed for complicated models to fine-tune the hyper-parameters sets with a great amount of data; and 4) "regularization" strategy is proposed to guarantee models performance to a great extent while dealing with real world issues by feature-selection, and by distinguishing more useful and less useful features.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Despite the rapid progress of ML methods, there are several potential flaws, including data bias, overfitting, resource-intensive training, and limited transfer learning.
“Despite the rapid progress of ML methods, there are several potential flaws, including data bias,27 overfitting,28 resource-intensive training,29 and limited transfer learning.30”
From Artificial intelligence in diabetes management: Advancements, opportunities, and challenges (2023) · cites this paper as [28]
Cited by
Sorted by citations
Artificial intelligence in diabetes management: Advancements, opportunities, and challenges
Guan, Z., Li, H., Liu, R., Cai, C., Liu, Y., Li, J., Wang, X., Huang, S., Wu, L., Liu, D., et al. (2023) · Cell Reports Medicine · doi:10.1016/j.xcrm.2023.101213 · Cited by 287 · see citing works
Spectral images based environmental sound classification using CNN with meaningful data augmentation
Mushtaq, Z., Su, S.-F., & Tran, Q.-V. (2021) · Applied Acoustics · doi:10.1016/j.apacoust.2020.107581 · Cited by 188 · see citing works
Cross-Validation Visualized: A Narrative Guide to Advanced Methods
Allgaier, J., & Pryss, R. (2024) · Machine Learning and Knowledge Extraction · doi:10.3390/make6020065 · Cited by 176 · see citing works
Kammarchedu, V., Butler, D., & Ebrahimi, A. (2022) · Analytica Chimica Acta · doi:10.1016/j.aca.2022.340447 · Cited by 79 · see citing works
Sharing Biomedical Data: Strengthening AI Development in Healthcare
Pereira, T., Morgado, J., Silva, F., Pelter, M. M., Dias, V. R., Barros, R., Freitas, C., Negrão, E., Flor de Lima, B., Correia da Silva, M., et al. (2021) · Healthcare · doi:10.3390/healthcare9070827 · Cited by 54 · see citing works
“With small amounts of data, models will not be able to create a boundary between classes with good accuracy on unseen data and may be biased by some characteristics of the training set [12].”
BackgroundShobayo, O., & Saatchi, R. (2025) · Diagnostics · doi:10.3390/diagnostics15091072 · Cited by 53 · see citing works
Forecasting Energy Consumption of a Public Building Using Transformer and Support Vector Regression
Huang, J., & Kaewunruen, S. (2023) · Energies · doi:10.3390/en16020966 · Cited by 38 · see citing works
Liability Regimes in the Age of AI: a Use-Case Driven Analysis of the Burden of Proof
Fernández Llorca, D., Charisi, V., Hamon, R., Sánchez, I., & Gómez, E. (2023) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.14565 · Cited by 20 · see citing works
Edge AI System Using a Thermal Camera for Industrial Anomaly Detection
Oliveira, V. M., & Moreira, A. H. J. (2022) · Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering · doi:10.1007/978-3-031-06371-8_12 · Cited by 10 · see citing works
Kim, B., Youm, C., Park, H., Choi, H., & Shin, S. (2024) · Frontiers in Public Health · doi:10.3389/fpubh.2024.1376736 · Cited by 4 · see citing works
Improving Generative Ai Reliability: A Qualitative Study of Hallucination in Large Language Models
Gwon, H. J., Yoo, Y., & Lee, J. Y. (2025) · doi:10.2139/ssrn.5118420 · Cited by 4 · see citing works
AI-based FPGA Accelerator for EVs Battery Management System
Nagarale, S. D., & Patil, B. P. (2023) · doi:10.21203/rs.3.rs-2963605/v1 · Cited by 16 · see citing works
- Authors
- Ying, X.
- Venue
- Journal of Physics: Conference Series
- Publisher
- IOP Publishing
- Published
- 2019-02
- Volume
- 1168
- Pages
- 022022
- ISSN
- 1742-6588 (ISSN-L)
- 1742-6596
- OpenAlex
- W2922073769
- Cited by
- 2,352
- References
- 0
- Citations by year
- 2026: 170 · 2025: 538 · 2024: 634 · 2023: 508 · 2022: 303 · 2021: 174 · 2020: 23 · 2019: 2
- FWCI
- 48.12
- Type
- conference paper
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- Locations
- Journal of Physics Conference Series (publishedVersion, cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Algorithms
- Physical Sciences › Computer Science › Artificial Intelligence › Imbalanced Data Classification Techniques
- Keywords
- overfitting, generalization, early stopping, network pruning, data expansion, hyperparameter optimization, regularization, feature selection, noisy training data, model complexity
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
