Overfitting Avoidance as Bias

Classification and Regression Trees.

Gordon, A. D., Breiman, L., Friedman, J. H., Olshen, R. A., & Stone, C. J. (1984) · Biometrics · doi:10.2307/2530946 · Cited by 23,867

Induction of decision trees

Quinlan, J. R. (1986) · Machine Learning · doi:10.1007/bf00116251 · Cited by 21,439

Classification and Regression Trees.

Ryzin, J. V., Breiman, L., Friedman, J. H., Olshen, R. A., & Stone, C. J. (1986) · Journal of the American Statistical Association · doi:10.2307/2288003 · Cited by 21,030

Classification And Regression Trees

Breiman, L., Friedman, J. H., Olshen, R. A., & Stone, C. J. (2017) · doi:10.1201/9781315139470 · Cited by 11,417

Practical Nonparametric Statistics

Groggel, D. J., & Conover, W. J. (2000) · Technometrics · doi:10.2307/1271101 · Cited by 11,076

Applied Nonparametric Statistics

Wozniak, P. J., & Daniel, W. W. (1991) · Technometrics · doi:10.2307/1268795 · Cited by 5,008

A Theory of the Learnable

Valiant, L. G. (1984) · ACM eBooks · doi:10.1145/3749740.3749746 · Cited by 4,211

A theory of the learnable

Valiant, L. G. (1984) · Communications of the ACM · doi:10.1145/1968.1972 · Cited by 3,279

Untitled work

(2013) · Springer eBooks · Cited by 2,625

Simplifying decision trees

Quinlan, J. R. (1987) · International Journal of Man-Machine Studies · doi:10.1016/s0020-7373(87)80053-6 · Cited by 2,553

The use of the area under the ROC curve in the evaluation of machine learning algorithms

Bradley, A. P. (1997) · Pattern Recognition · doi:10.1016/s0031-3203(96)00142-2 · Cited by 7,217

Semi-Supervised Learning

Chapelle, O., Schölkopf, B., & Zien, A. (2006) · The MIT Press eBooks · doi:10.7551/mitpress/9780262033589.001.0001 · Cited by 4,642

Feature Selection for Classification

Dash, M., & Liu, H. (1997) · Intelligent Data Analysis · doi:10.1016/s1088-467x(97)00008-5 · Cited by 3,594

The class imbalance problem: A systematic study1

Japkowicz, N., & Stephen, S. (2002) · Intelligent Data Analysis · doi:10.3233/ida-2002-6504 · Cited by 3,295

Very Simple Classification Rules Perform Well on Most Commonly Used Datasets

Holte, R. C. (1993) · Machine Learning · doi:10.1023/a:1022631118932 · Cited by 1,800

The Lack of A Priori Distinctions Between Learning Algorithms

Wolpert, D. H. (1996) · Neural Computation · doi:10.1162/neco.1996.8.7.1341 · Cited by 1,652

Improved Heterogeneous Distance Functions

Wilson, D. R., & Martinez, T. R. (1997) · Journal of Artificial Intelligence Research · doi:10.1613/jair.346 · Cited by 1,350

Reduction Techniques for Instance-Based Learning Algorithms

Wilson, D. R., & Martinez, T. R. (2000) · Machine Learning · doi:10.1023/a:1007626913721 · Cited by 1,182

Automatic Construction of Decision Trees from Data: A Multi-Disciplinary Survey

Murthy, S. K. (1998) · Data Mining and Knowledge Discovery · doi:10.1023/a:1009744630224 · Cited by 1,085

Data mining with an ant colony optimization algorithm

Parpinelli, R. S., Lopes, H. S., & Freitas, A. A. (2002) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2002.802452 · Cited by 985

Class Noise vs. Attribute Noise: A Quantitative Study

Xingquan Zhu, A., & Wu, X. (2004) · Artificial Intelligence Review · doi:10.1007/s10462-004-0751-8 · Cited by 896

How persuasive is a good fit? A comment on theory testing.

Roberts, S., & Pashler, H. (2000) · Psychological Review · doi:10.1037/0033-295x.107.2.358 · Cited by 879

Automated reverse engineering of nonlinear dynamical systems

Bongard, J., & Lipson, H. (2007) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0609476104 · Cited by 792

Knowledge-based artificial neural networks

Towell, G. G., & SHAVLIK, J. W. (1994) · Artificial Intelligence · doi:10.1016/0004-3702(94)90105-8 · Cited by 770

A System for Induction of Oblique Decision Trees

Murthy, S. K., Kasif, S., & Salzberg, S. L. (1994) · Cited by 723

Fuzzy decision trees: issues and methods

Janikow, C. Z. (1998) · IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics) · doi:10.1109/3477.658573 · Cited by 659

Object-based crop identification using multiple vegetation indices, textural features and crop phenology

Peña, J. M., Ngugi, M. K., Plant, R. E., & Six, J. (2011) · Remote Sensing of Environment · doi:10.1016/j.rse.2011.01.009 · Cited by 631

A System for Induction of Oblique Decision Trees

Murthy, S. K., Kasif, S., & Salzberg, S. L. (1994) · Journal of Artificial Intelligence Research · doi:10.1613/jair.63 · Cited by 890

A comparative analysis of methods for pruning decision trees

Esposito, F., Malerba, D., Semeraro, G. M., & Kay, J. (1997) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.589207 · Cited by 551

Separate-and-Conquer Rule Learning

Fürnkranz, J. (1999) · Artificial Intelligence Review · doi:10.1023/a:1006524209794 · Cited by 531

Prototype and Feature Selection by Sampling and Random Mutation Hill Climbing Algorithms

Skalak, D. B. (1994) · Elsevier eBooks · doi:10.1016/b978-1-55860-335-6.50043-x · Cited by 487

The Role of Occam's Razor in Knowledge Discovery

Domingos, P. (1999) · Data Mining and Knowledge Discovery · doi:10.1023/a:1009868929893 · Cited by 459

Learning from Imbalanced Data Sets: A Comparison of Various Strategies *

Japkowicz, N. (2000) · Cited by 455

A Survey of Evolutionary Algorithms for Data Mining and Knowledge Discovery

Freitas, A. A. (2003) · Natural computing series · doi:10.1007/978-3-642-18965-4_33 · Cited by 425

Incremental Reduced Error Pruning

Fürnkranz, J., & Widmer, G. (1994) · Elsevier eBooks · doi:10.1016/b978-1-55860-335-6.50017-9 · Cited by 371

Simplifying decision trees: A survey

Breslow, L. A., & Aha, D. W. (1997) · The Knowledge Engineering Review · doi:10.1017/s0269888997000015 · Cited by 344

A Conservation Law for Generalization Performance

Schaffer, C. (1994) · Elsevier eBooks · doi:10.1016/b978-1-55860-335-6.50039-8 · Cited by 340

Methods to Avoid Over-Fitting and Under-Fitting in Supervised Machine Learning (Comparative Study)

Jabbar, H. K., & Khan, R. Z. (2014) · International Conference on Computer Science, Communication and Instrumentation Devices · doi:10.3850/978-981-09-5247-1_017 · Cited by 332

Evolving fuzzy and neuro-fuzzy approaches in clustering, regression, identification, and classification: A Survey

Škrjanc, I., Iglesias, J. A., Sanchis, A., Leite, D. F., Lughofer, E. D., & Gomide, F. A. C. (2019) · Information Sciences · doi:10.1016/j.ins.2019.03.060 · Cited by 329

Mathematical Programming for Data Mining: Formulations and Challenges

Bradley, P., Fayyad, U. M., & Mangasarian, O. L. (1999) · INFORMS journal on computing · doi:10.1287/ijoc.11.3.217 · Cited by 248

Multiple Comparisons in Induction Algorithms

Jensen, D. D., & Cohen, P. R. (2000) · Machine Learning · doi:10.1023/a:1007631014630 · Cited by 241

Tracking the Evolution of Smartphone Sensing for Monitoring Human Movement

Rosario, M. D., Redmond, S. J., & Lovell, N. H. (2015) · Sensors · doi:10.3390/s150818901 · Cited by 216

Comparison of machine learning models for gully erosion susceptibility mapping

Arabameri, A., Chen, W., Loche, M., Zhao, X., Li, Y., Lombardo, L., Cerdà, A., Pradhan, B., & Bui, D. T. (2019) · Geoscience Frontiers · doi:10.1016/j.gsf.2019.11.009 · Cited by 177

Using long short-term memory recurrent neural network in land cover classification on Landsat and Cropland data layer time series

Sun, Z., Di, L., & Fang, H. (2018) · International Journal of Remote Sensing · doi:10.1080/01431161.2018.1516313 · Cited by 187

Groundwater Potential Mapping Using Remote Sensing and GIS-Based Machine Learning Techniques

Lee, S., Hyun, Y., Lee, S., & Lee, M.-J. (2020) · Remote Sensing · doi:10.3390/rs12071200 · Cited by 175

Decision trees in epidemiological research

Venkatasubramaniam, A., Wolfson, J. A., Mitchell, N. R., Barnes, T. L., JaKa, M. M., & French, S. A. (2017) · Emerging Themes in Epidemiology · doi:10.1186/s12982-017-0064-4 · Cited by 164

Selecting a classification method by cross-validation

Schaffer, C. (1993) · Machine Learning · doi:10.1023/a:1022639714137 · Cited by 476

Understanding the Crucial Role of Attribute Interaction in Data Mining

Freitas, A. A. (2001) · Artificial Intelligence Review · doi:10.1023/a:1011996210207 · Cited by 157

Human-in-the-loop Bayesian optimization of wearable device parameters

Kim, M., Ding, Y., Malcolm, P., Speeckaert, J., Siviy, C., Walsh, C. J., & Kuindersma, S. (2017) · PLoS ONE · doi:10.1371/journal.pone.0184054 · Cited by 155

The nearest subclass classifier: a compromise between the nearest mean and nearest neighbor classifier

Veenman, C. J., & Reinders, M. J. T. (2005) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2005.187 · Cited by 148

Discovering Neural Nets with Low Kolmogorov Complexity and High Generalization Capability

Schmidhuber, J. (1997) · Neural Networks · doi:10.1016/s0893-6080(96)00127-x · Cited by 148

Syntactic Measures of Complexity

Edmonds, B. (1999) · Cited by 138

Pruning Algorithms for Rule Learning

Fürnkranz, J. (1997) · Machine Learning · doi:10.1023/a:1007329424533 · Cited by 137

Mathematical Programming in Data Mining

Mangasarian, O. L. (1997) · Data Mining and Knowledge Discovery · doi:10.1023/a:1009735908398 · Cited by 130

Technical note: Bias and the quantification of stability

Turney, P. D. (1995) · Machine Learning · doi:10.1007/bf00993473 · Cited by 121

Oversearching and layered search in empirical learning

Quinlan, J. R., & Cameron-Jones, R. M. (1995) · Cited by 119

Automatic Diabetic Retinopathy Diagnosis Using Adaptive Fine-Tuned Convolutional Neural Network

Saeed, F., Hussain, M., & Aboalsamh, H. (2021) · IEEE Access · doi:10.1109/access.2021.3065273 · Cited by 112

Further Experimental Evidence against the Utility of Occam's Razor

Webb, G. I. (1996) · Journal of Artificial Intelligence Research · doi:10.1613/jair.228 · Cited by 108

A Comparison of Traditional Machine Learning and Deep Learning in Image Recognition

Lai, Y. (2019) · Journal of Physics Conference Series · doi:10.1088/1742-6596/1314/1/012148 · Cited by 107

Noise modelling and evaluating learning from examples

Hickey, R. J. (1996) · Artificial Intelligence · doi:10.1016/0004-3702(94)00094-8 · Cited by 105

Semi-Supervised Learning

Adams, N. M. (2009) · Journal of the Royal Statistical Society Series A (Statistics in Society) · doi:10.1111/j.1467-985x.2009.00588_3.x · Cited by 113

Understanding the crucial differences between classification and discovery of association rules

Freitas, A. A. (2000) · ACM SIGKDD Explorations Newsletter · doi:10.1145/360402.360423 · Cited by 100

Exploring the Decision Forest: An Empirical Investigation of Occam's Razor in Decision Tree Induction

Murphy, P. M., & Pazzani, M. J. (1994) · Journal of Artificial Intelligence Research · doi:10.1613/jair.41 · Cited by 99

The no-free-lunch theorems of supervised learning

Sterkenburg, T. F., & Grünwald, P. (2021) · Synthese · doi:10.1007/s11229-021-03233-1 · Cited by 103

River/stream water temperature forecasting using artificial intelligence models: a systematic review

Zhu, S., & Piotrowski, A. (2020) · Acta Geophysica · doi:10.1007/s11600-020-00480-7 · Cited by 91

Histopathology classification and localization of colorectal cancer using global labels by weakly supervised deep learning

Zhou, C., Jin, Y., Chen, Y., Huang, S., Huang, R., Wang, Y., Zhao, Y., Yao, C., Guo, L., & Liao, J. (2021) · Computerized Medical Imaging and Graphics · doi:10.1016/j.compmedimag.2021.101861 · Cited by 86

Inductive Logic Programming At 30: A New Introduction

Cropper, A., & Dumančić, S. (2022) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.13507 · Cited by 83

The development of fuzzy decision trees in the framework of Axiomatic Fuzzy Set logic

Liu, X., & Pedrycz, W. (2005) · Applied Soft Computing · doi:10.1016/j.asoc.2005.07.003 · Cited by 83

An introduction to machine learning and analysis of its use in rheumatic diseases

Kingsmore, K. M., Puglisi, C., Grammer, A. C., & Lipsky, P. E. (2021) · Nature Reviews Rheumatology · doi:10.1038/s41584-021-00708-w · Cited by 86

A review of machine learning in geochemistry and cosmochemistry: Method improvements and applications

He, Y., Zhou, Y., Wen, T., Zhang, S., Huang, F., Zou, X., Ma, X., & Zhu, Y. (2022) · Applied Geochemistry · doi:10.1016/j.apgeochem.2022.105273 · Cited by 87

Occam's two razors: the sharp and the blunt

Domingos, P. (1998) · Cited by 82

Machine Learning for Silver Nanoparticle Electron Transfer Property Prediction

Sun, B., Fernández, M., & Barnard, A. S. (2017) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.7b00272 · Cited by 79

MDL and Categorical Theories (Continued)

Quinlan, J. R. (1995) · Elsevier eBooks · doi:10.1016/b978-1-55860-377-6.50064-5 · Cited by 77

Gene selection and classification using Taguchi chaotic binary particle swarm optimization

Chuang, L., Yang, C., Wu, K., & Yang, C. (2011) · Expert Systems with Applications · doi:10.1016/j.eswa.2011.04.165 · Cited by 75

Mapping sugarcane in complex landscapes by integrating multi-temporal Sentinel-2 images and machine learning algorithms

Wang, M., Liu, Z., Baig, M. H. A., Wang, Y., Li, Y., & Chen, Y. (2019) · Land Use Policy · doi:10.1016/j.landusepol.2019.104190 · Cited by 74

An Exact Probability Metric for Decision Tree Splitting and Stopping

Martin, J. K. (1997) · Machine Learning · doi:10.1023/a:1007367629006 · Cited by 72

Dynamic Integration of Regression Models

Rooney, N., Patterson, D., Anand, S. S., & Tsymbal, A. (2004) · Lecture notes in computer science · doi:10.1007/978-3-540-25966-4_16 · Cited by 69

Evolutionary Processes and Organizational Adaptation

Levinthal, D. A. (2021) · doi:10.1093/oso/9780199684946.001.0001 · Cited by 65

Cross validation and MLP architecture selection

Andersen, T., & Martinez, T. R. (2003) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.1999.832613 · Cited by 61

Demand forecasting for fashion products: A systematic review

Swaminathan, K., & Venkitasubramony, R. (2023) · International Journal of Forecasting · doi:10.1016/j.ijforecast.2023.02.005 · Cited by 60

Pruning Decision Trees and Lists

Frank, E. (2000) · Cited by 59

For Every Generalization Action, Is There Really an Equal and Opposite Reaction? Analysis of the Conservation Law for Generalization Performance

Rao, R. B., Gordon, D. F., & Spears, W. (1995) · Elsevier eBooks · doi:10.1016/b978-1-55860-377-6.50065-7 · Cited by 64

A hybrid nearest-neighbor and nearest-hyperrectangle algorithm

Wettschereck, D. (1994) · Lecture notes in computer science · doi:10.1007/3-540-57868-4_67 · Cited by 56

Induction of comprehensible models for gene expression datasets by subgroup discovery methodology

Gamberger, D., Lavrač, N., Železný, F., & Tolar, J. (2004) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2004.07.007 · Cited by 54

Anomaly-Based Intrusion Detection by Machine Learning: A Case Study on Probing Attacks to an Institutional Network

Tufan, E., Tezcan, C., & Acartürk, C. (2021) · IEEE Access · doi:10.1109/access.2021.3068961 · Cited by 53

GA Tree: genetically evolved decision trees

Papagelis, A., & Kalles, D. (2002) · IEEE International Conference on Tools with Artificial Intelligence (ICTAI) · doi:10.1109/tai.2000.889871 · Cited by 50

Generalized versus non-generalized neural network model for multi-lead inflow forecasting at Aswan High Dam

El‐Shafie, A., & Noureldin, A. M. (2011) · Hydrology and earth system sciences · doi:10.5194/hess-15-841-2011 · Cited by 49

Data mining using genetic programming: the implications of parsimony on generalization error

Cavaretta, M. J., & Chellapilla, K. (2003) · IEEE Congress on Evolutionary Computation (CEC) · doi:10.1109/cec.1999.782602 · Cited by 46

A Survey on Traffic Prediction Techniques Using Artificial Intelligence for Communication Networks

Chen, A., Law, J., & Aibin, M. (2021) · Telecom · doi:10.3390/telecom2040029 · Cited by 47

Feature subset selection based on fuzzy entropy measures for handling classification problems

Shie, J.-D., & Chen, S. (2007) · Applied Intelligence · doi:10.1007/s10489-007-0042-6 · Cited by 47

A New Metric-Based Approach to Model Selection

Schuurmans, D. (1997) · Cited by 45

BEXA: A Covering Algorithm for Learning Propositional Concept Descriptions

Theron, H., & Cloete, I. (1996) · Machine Learning · doi:10.1007/bf00117830 · Cited by 44

Metric-Based Methods for Adaptive Model Selection and Regularization

Schuurmans, D., & Southey, F. (2002) · Machine Learning · doi:10.1023/a:1013947519741 · Cited by 44

A Further Comparison of Simplification Methods for Decision-Tree Induction

Malerba, D., Esposito, F., & Semeraro, G. M. (1996) · Lecture notes in statistics · doi:10.1007/978-1-4612-2404-4_35 · Cited by 43

Associative Reinforcement Learning: Functions in k-DNF

Kaelbling, L. P. (1994) · Machine Learning · doi:10.1023/a:1022689909846 · Cited by 46

Neuropsychiatric symptoms as predictors of conversion from MCI to dementia: a machine learning approach

Mallo, S. C., Valladares‐Rodríguez, S., Facal-Mayo, D., Lojo‐Seoane, C., Iglesias, M. J. F., & Pereiro, A. X. (2019) · International Psychogeriatrics · doi:10.1017/s1041610219001030 · Cited by 43

On Overfitting Avoidance as Bias

Wolpert, D. H. (1993) · Cited by 42

The Minimum Description Length Principle and Categorical Theories

Quinlan, J. R. (1994) · Elsevier eBooks · doi:10.1016/b978-1-55860-335-6.50036-2 · Cited by 40

Simplest Rules Characterizing Classes Generated by δ-Free Sets

Crémilleux, B., & Boulicaut, J. (2003) · Springer eBooks · doi:10.1007/978-1-4471-0651-7_3 · Cited by 38

Few-Shot Scene Classification of Optical Remote Sensing Images Leveraging Calibrated Pretext Tasks

Ji, H., Gao, Z. Y., Zhang, Y., Wan, Y., Li, C., & Mei, T. (2022) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2022.3184080 · Cited by 39

Online bagging of evolving fuzzy systems

Lughofer, E. D., Pratama, M., & Škrjanc, I. (2021) · Information Sciences · doi:10.1016/j.ins.2021.04.041 · Cited by 35

Error Estimation and Model Selection

Scheffer, T. (1999) · Cited by 35

Parsimonious Least Norm Approximation

Bradley, P., Mangasarian, O. L., & Rosen, J. B. (1998) · Computational Optimization and Applications · doi:10.1023/a:1018361916442 · Cited by 34

FOSSIL: A robust relational learner

Fürnkranz, J. (1994) · Lecture notes in computer science · doi:10.1007/3-540-57868-4_54 · Cited by 33

Predicting travel mode choice with a robust neural network and Shapley additive explanations analysis

Tang, L., Tang, C., Fu, Q., & Ma, C. (2024) · IET Intelligent Transport Systems · doi:10.1049/itr2.12514 · Cited by 32

DeepRisk: A Deep Transfer Learning Approach to Migratable Traffic Risk Estimation in Intelligent Transportation Using Social Sensing

Zhang, Y., Wang, H., Zhang, D., & Wang, D. (2019) · International Conference on Distributed Computing in Smart Systems and the Internet of Things (DCOSS-IoT) · doi:10.1109/dcoss.2019.00039 · Cited by 32

Efficient generation of accurate mobility maps using machine learning algorithms

Mechergui, D., & Jayakumar, P. (2020) · Journal of Terramechanics · doi:10.1016/j.jterra.2019.12.002 · Cited by 31

In Defense of C4.5: Notes on Learning One-Level Decision Trees

Elomaa, T. (1994) · Elsevier eBooks · doi:10.1016/b978-1-55860-335-6.50016-7 · Cited by 31

NONLINEAR EFFECTS OF FEMALE MATE CHOICE IN WILD THREESPINE STICKLEBACKS

Blais, J., Bernatchez, L., & Rico, C. (2004) · Evolution · doi:10.1111/j.0014-3820.2004.tb00879.x · Cited by 31

DOGMA: A GA-based relational learner

Hekanaho, J. (1998) · Lecture notes in computer science · doi:10.1007/bfb0027324 · Cited by 30

Sparse data and the effect of overfitting avoidance in decision tree induction

Schaffer, C. (1992) · Cited by 24

Deconstructing the Digit Recognition Problem

Schaffer, C. (1992) · Elsevier eBooks · doi:10.1016/b978-1-55860-247-2.50056-5 · Cited by 17

Adjusting for multiple testing in decision tree pruning

Jensen, D. (1996)

Avoiding Overfitting of Decision Trees

Bramer, M. (2020) · Undergraduate topics in computer science · doi:10.1007/978-1-4471-7493-6_9 · Cited by 14

When does overfitting decrease prediction accuracy in induced decision trees and rule sets?

Schaffer, C. (1991) · Lecture notes in computer science · doi:10.1007/bfb0017014 · Cited by 25