Benchmarking state-of-the-art classification algorithms for credit scoring
The DOI matches its Crossref record; 33 of the 33 references it cites were found in a registry or an index.
Abstract
In this paper, we study the performance of various state-of-the-art classification algorithms applied to eight real-life credit scoring data sets. Some of the data sets originate from major Benelux and UK financial institutions. Different types of classifiers are evaluated and compared. Besides the well-known classification algorithms (eg logistic regression, discriminant analysis, k-nearest neighbour, neural networks and decision trees), this study also investigates the suitability and performance of some recently proposed, advanced kernel-based classification algorithms such as support vector machines and least-squares support vector machines (LS-SVMs). The performance is assessed using the classification accuracy and the area under the receiver operating characteristic curve. Statistically significant performance differences are identified using the appropriate test statistics. It is found that both the LS-SVM and neural network classifiers yield a very good performance, but also simple classifiers such as logistic regression and linear discriminant analysis perform very well for credit scoring.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Many benchmarking studies have illustrated that complex analytical techniques only provide marginal performance gains on structured, tabular data sets as frequently encountered in common classification tasks such as fraud detection, credit scoring and marketing analytics (Baesens et al., 2003; Lessmann et al., 2015).”
From Data engineering for fraud detection (2021) · cites this paper as Baesens et al., 2003
In the citing paper
“We would also like to note that these complex models not always significantly outperform simple analytical models such as logistic regression (Baesens et al., 2003; Lessmann et al., 2015)”
From Data engineering for fraud detection (2021) · cites this paper as Baesens et al., 2003
Citations
Cites
Wu, Y. (1999) · Technometrics · doi:10.1080/00401706.1999.10485951 · Cited by 27,221
The Elements of Statistical Learning
Hastie, T., Tibshirani, R., & Friedman, J. (2009) · Springer Series in Statistics · doi:10.1007/978-0-387-84858-7 · Cited by 25,970
(1992) · Cited by 23,431
DeLong, E. R., DeLong, D. M., & Clarke-Pearson, D. L. (1988) · Biometrics · doi:10.2307/2531595 · Cited by 22,649
Neural networks for pattern recognition
(1994) · Choice Reviews Online · doi:10.5860/choice.31-5500 · Cited by 18,538
The Elements of Statistical Learning: Data Mining, Inference, and Prediction
John Lu, Z. Q. (2010) · Journal of the Royal Statistical Society Series A: Statistics in Society · doi:10.1111/j.1467-985x.2010.00646_6.x · Cited by 12,390
Neural Networks for Pattern Recognition
Bishop, C. M. (1995) · doi:10.1093/oso/9780198538493.001.0001 · Cited by 12,287
Least Squares Support Vector Machine Classifiers
Suykens, J. A. K., & Vandewalle, J. (1999) · Neural Processing Letters · doi:10.1023/a:1018628609742 · Cited by 9,362
Self-citationFriedman, N., Geiger, D., & Goldszmidt, M. (1997) · Machine Learning · doi:10.1023/a:1007465528199 · Cited by 4,742
Handbook of Parametric and Nonparametric Statistical Procedures
(2004) · Technometrics · doi:10.1198/tech.2004.s209 · Cited by 4,661
Least Squares Support Vector Machines
Suykens, J. A. K., Van Gestel, T., De Brabanter, J., De Moor, B., & Vandewalle, J. (2002) · WORLD SCIENTIFIC eBooks · doi:10.1142/5089 · Cited by 3,642
Self-citationEstimating Continuous Distributions in Bayesian Classifiers
John, G. H., & Langley, P. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1302.4964 · Cited by 2,885
Multi-Interval Discretization of Continuous-Valued Attributes for Classification Learning
Fayyad, U. M., & Irani, K. B. (1993) · International Joint Conference on Artificial Intelligence · Cited by 2,870
Signal detection theory and ROC analysis
Egan, J. P. (1975) · Academic Press eBooks · Cited by 1,659
(1982) · Cited by 1,288
Neural network credit scoring models
West, D. (2000) · Computers & Operations Research · doi:10.1016/s0305-0548(99)00149-5 · Cited by 1,033
The Case against Accuracy Estimation for Comparing Induction Algorithms
Provost, F. J., Fawcett, T. E., & Kohavi, R. (1998) · Cited by 1,031
Fawcett, T., & Provost, F. (1997) · Data Mining and Knowledge Discovery · doi:10.1023/a:1009700419189 · Cited by 881
Benchmarking Least Squares Support Vector Machine Classifiers
van Gestel, T., Suykens, J. A. K., Baesens, B., Viaene, S., Vanthienen, J., Dedene, G., de Moor, B., & Vandewalle, J. (2004) · Machine Learning · doi:10.1023/b:mach.0000008082.80494.e0 · Cited by 745
Self-citationA survey of credit and behavioural scoring: forecasting financial risk of lending to consumers
Thomas, L. C. (2000) · International Journal of Forecasting · doi:10.1016/s0169-2070(00)00034-0 · Cited by 741
The Evidence Framework Applied to Classification Networks
MacKay, D. J. C. (1992) · Neural Computation · doi:10.1162/neco.1992.4.5.720 · Cited by 722
Credit Scoring and Its Applications
Thomas, L. C., Edelman, D. B., & Crook, J. N. (2002) · Society for Industrial and Applied Mathematics eBooks · doi:10.1137/1.9780898718317 · Cited by 667
Handbook of Parametric and Nonparametric Statistical Procedures
Nash, M. S. (2001) · Technometrics · doi:10.1198/tech.2001.s629 · Cited by 570
A comparison of neural networks and linear scoring models in the credit union environment
Desai, V. S., Crook, J. N., & Overstreet, G. A. (1996) · European Journal of Operational Research · doi:10.1016/0377-2217(95)00246-4 · Cited by 522
Credit Scoring and Its Applications
Thomas, L. C., Crook, J. N., & Edelman, D. B. (2017) · Cited by 308
Coast, J. (1997) · BMJ · doi:10.1136/bmj.314.7087.1118 · Cited by 252
Bayesian neural network learning for repeat purchase modelling in direct marketing
Baesens, B., Viaene, S., Van den Poel, D., Vanthienen, J., & Dedene, G. (2002) · European Journal of Operational Research · doi:10.1016/s0377-2217(01)00129-1 · Cited by 218
Self-citationCredit scoring using neural and evolutionary techniques
Yobas, M. B. (2000) · IMA Journal of Management Mathematics · doi:10.1093/imaman/11.2.111 · Cited by 186
A credit scoring model for personal loans
Steenackers, A., & Goovaerts, M. J. (1989) · Insurance Mathematics and Economics · doi:10.1016/0167-6687(89)90044-9 · Cited by 183
Construction of a k-nearest-neighbour credit-scoring system
Henley, W. (1997) · IMA Journal of Management Mathematics · doi:10.1093/imaman/8.4.305 · Cited by 112
PHAB scores: proportional hazards analysis behavioural scores
Stepanova, M., & Thomas, L. C. (2001) · Journal of the Operational Research Society · doi:10.1057/palgrave.jors.2601189 · Cited by 95
Self-citationClassification accuracy in discriminant analysis: a mixed integer programming approach
Glen, J. J. (2001) · Journal of the Operational Research Society · doi:10.1057/palgrave.jors.2601085 · Cited by 39
Does scoring a subpopulation make a difference
Banasik, J., Crook, J. N., & Thomas, L. C. (1996) · The International Review of Retail Distribution and Consumer Research · doi:10.1080/09593969600000019 · Cited by 30
Research works printed but not linked: 3
- Henley WE, 1997, IMA J Math Appl Bus Ind, 8, 305
- Yobas MB, 2000, IMA J Math Appl Bus Ind, 11, 111
- Banasik J, 1996, Int Rev Ret Dis Cons Res, 6, 180
Cited by
Benchmarking state-of-the-art classification algorithms for credit scoring: an update of research
Lessmann, S., Baesens, B., Seow, H.-V., & Thomas, L. C. (2015) · European Journal of Operational Research · doi:10.1016/j.ejor.2015.05.030 · Cited by 1,284
“We pay special attention to ensemble classifiers, which Baesens, et al. (2003) do not cover.”
BackgroundMethodResultInfluentialSelf-citationLessmann, S., Baesens, B., Mues, C., & Pietsch, S. (2008) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2008.35 · Cited by 1,226
“It is known that a careful tuning of such hyperparameters is essential to obtain a representative assessment of the classifier’s potential (see, e.g., [3], [63]).”
BackgroundSelf-citationYeh, I.-C., & Lien, C.-H. (2009) · Expert Systems with Applications · doi:10.1016/j.eswa.2007.12.020 · Cited by 885
“Baesens et al. (2003) found that, based on eight real-life credit scoring data sets, both the LS-SVM and neural network classifiers yield a very good performance, but also simple classifiers such as logistic regression and linear discriminant analysis perform very well for credit scoring.”
BackgroundCredit scoring with a data mining approach based on support vector machines
Huang, C.-L., Chen, M.-C., & Wang, C.-J. (2007) · Expert Systems with Applications · doi:10.1016/j.eswa.2006.07.007 · Cited by 804
A boosted decision tree approach using Bayesian hyper-parameter optimization for credit scoring
Xia, Y., Liu, C., Li, Y., & Liu, N. (2017) · Expert Systems with Applications · doi:10.1016/j.eswa.2017.02.017 · Cited by 769
“SVM requires appropriate tuning of parameters (Huang et al., 2007), and tasks are AC CE PT ED M AN US CR IP T handled by GS (Baesens et al., 2003).”
Method
Metadata
- Authors
- Baesens, B., Van Gestel, T., Viaene, S., Stepanova, M., Suykens, J., & Vanthienen, J.
- Affiliations
- KU Leuven (BE)
- K.U.Leuven Leuven Belgium
- Union Bank of Switzerland (CH)
- Corresponding author
- Baesens, B
- Viaene, S
- Venue
- Journal of the Operational Research Society
- Publisher
- Informa UK Limited
- Published
- 2003-06
- Volume
- 54
- Issue
- 6
- Pages
- 627-635
- ISSN
- 0160-5682 (ISSN-L)
- 1476-9360
- OpenAlex
- W1982120517
- Cited by
- 904
- References
- 33
- Citations by year
- 2026: 54 · 2025: 51 · 2024: 41 · 2023: 52 · 2022: 60 · 2021: 42 · 2020: 57 · 2019: 65 · 2018: 41 · 2017: 42 · 2016: 45 · 2015: 43 · 2014: 26 · 2013: 25 · 2012: 35
- FWCI
- 50.33
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Journal of the Operational Research Society (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- ePrints Soton (University of Southampton) (submittedVersion)
- Topics
- Social Sciences › Business, Management and Accounting › Accounting › Financial Distress and Bankruptcy Prediction
- Physical Sciences › Computer Science › Artificial Intelligence › Imbalanced Data Classification Techniques
- Physical Sciences › Computer Science › Information Systems › Data Mining and Machine Learning Applications
- Keywords
- credit scoring, machine learning classifiers, benchmarking, support vector machine, least squares support vector machine, artificial neural networks, logistic regression, linear discriminant analysis, k-nearest neighbors, decision tree, kernel-based classifiers, credit risk
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.