The DOI matches its Crossref record; 60 of the 60 references it cites were found in a registry or an index.
Dynamically Weighted Balanced Loss: Class Imbalanced Learning and Confidence Calibration of Deep Neural Networks
Open accessAbstract
Imbalanced class distribution is an inherent problem in many real-world classification tasks where the minority class is the class of interest. Many conventional statistical and machine learning classification algorithms are subject to frequency bias, and learning discriminating boundaries between the minority and majority classes could be challenging. To address the class distribution imbalance in deep learning, we propose a class rebalancing strategy based on a class-balanced dynamically weighted loss function where weights are assigned based on the class frequency and predicted probability of ground-truth class. The ability of dynamic weighting scheme to self-adapt its weights depending on the prediction scores allows the model to adjust for instances with varying levels of difficulty resulting in gradient updates driven by hard minority class samples. We further show that the proposed loss function is classification calibrated. Experiments conducted on highly imbalanced data across different applications of cyber intrusion detection (CICIDS2017 data set) and medical imaging (ISIC2019 data set) show robust generalization. Theoretical results supported by superior empirical performance provide justification for the validity of the proposed dynamically weighted balanced (DWB) loss function.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Focal Loss for Dense Object Detection
Lin, T.-Y., Goyal, P., Girshick, R., He, K., & Dollar, P. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.324 · Cited by 30,581
The Elements of Statistical Learning
Hastie, T., Tibshirani, R., & Friedman, J. (2009) · Springer Series in Statistics · doi:10.1007/978-0-387-84858-7 · Cited by 25,970
A decision-theoretic generalization of on-line learning and an application to boosting
Freund, Y., & Schapire, R. E. (1997) · Journal of Computer and System Sciences · doi:10.1006/jcss.1997.1504 · Cited by 20,466
On Information and Sufficiency
Kullback, S., & Leibler, R. A. (1951) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177729694 · Cited by 20,193
EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks
Tan, M., & Le, Q. V. (2019) · arXiv · doi:10.48550/arxiv.1905.11946 · Cited by 17,072
Breiman, L. (1996) · Machine Learning · doi:10.1007/bf00058655 · Cited by 16,325
Kuhn, M., & Johnson, K. (2013) · doi:10.1007/978-1-4614-6849-3 · Cited by 6,788
Large-Scale Machine Learning with Stochastic Gradient Descent
Bottou, L. (2010) · Proceedings of COMPSTAT'2010 · doi:10.1007/978-3-7908-2604-3_16 · Cited by 5,795
Toward Generating a New Intrusion Detection Dataset and Intrusion Traffic Characterization
Sharafaldin, I., Habibi Lashkari, A., & Ghorbani, A. A. (2018) · Proceedings of the 4th International Conference on Information Systems Security and Privacy · doi:10.5220/0006639801080116 · Cited by 5,318 · cited 1×
“1) Dataset Description: We rely on an intrusion detection dataset (CICIDS2017) published by the Canadian Institute for Cyber-Security (CIC) at the University of New Brunswick (UNB) [54].”
MethodADASYN: Adaptive synthetic sampling approach for imbalanced learning
He, H., Bai, Y., Garcia, E. A., & Li, S. (2008) · 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence) · doi:10.1109/ijcnn.2008.4633969 · Cited by 4,521
Borderline-SMOTE : A New Over-Sampling Method in Imbalanced Data Sets Learning
Han, H., Wang, W.-Y., & Mao, B.-H. (2005) · Lecture notes in computer science · doi:10.1007/11538059_91 · Cited by 4,331
Logistic Regression in Rare Events Data
King, G., & Zeng, L. (2001) · Political Analysis · doi:10.1093/oxfordjournals.pan.a004868 · Cited by 4,316 · cited 2×
“The parameter estimation bias under class imbalance also applies to models that typically produce calibrated probability estimates, such as logistic regression [45].”
MethodResultA study of the behavior of several methods for balancing machine learning training data
Batista, G. E. A. P. A., Prati, R. C., & Monard, M. C. (2004) · ACM SIGKDD Explorations Newsletter · doi:10.1145/1007730.1007735 · Cited by 4,203
Tschandl, P., Rosendahl, C., & Kittler, H. (2018) · Scientific Data · doi:10.1038/sdata.2018.161 · Cited by 3,628
On calibration of modern neural networks
Guo, C., Pleiss, G., Sun, Y., & Weinberger, K. Q. (2017) · arXiv · doi:10.48550/arxiv.1706.04599 · Cited by 3,435
The class imbalance problem: A systematic study1
Japkowicz, N., & Stephen, S. (2002) · Intelligent Data Analysis · doi:10.3233/ida-2002-6504 · Cited by 3,295
Class-Balanced Loss Based on Effective Number of Samples
Cui, Y., Jia, M., Lin, T.-Y., Song, Y., & Belongie, S. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00949 · Cited by 3,094
A systematic study of the class imbalance problem in convolutional neural networks
Buda, M., Maki, A., & Mazurowski, M. A. (2018) · Neural Networks · doi:10.1016/j.neunet.2018.07.011 · Cited by 3,088 · cited 1×
“Experiments conducted in [24] suggest that data sampling strategies have little effect on classification performance; however, results in [25] demonstrate that random oversampling leads to per-”
BackgroundSurvey on deep learning with class imbalance
Johnson, J. M., & Khoshgoftaar, T. M. (2019) · Journal of Big Data · doi:10.1186/s40537-019-0192-5 · Cited by 3,032
Galar, M., Fernandez, A., Barrenechea, E., Bustince, H., & Herrera, F. (2012) · IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews) · doi:10.1109/tsmcc.2011.2161285 · Cited by 2,794
Codella, N. C. F., Gutman, D., Celebi, M. E., Helba, B., Marchetti, M. A., Dusza, S. W., Kalloo, A., Liopyris, K., Mishra, N., Kittler, H., et al. (2018) · 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018) · doi:10.1109/isbi.2018.8363547 · Cited by 2,620
Editorial: special issue on learning from imbalanced data sets
Chawla, N. V., Japkowicz, N., & Kotcz, A. (2004) · ACM SIGKDD Explorations Newsletter · doi:10.1145/1007730.1007733 · Cited by 2,063
A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks
Hendrycks, D., & Gimpel, K. (2016) · arXiv · doi:10.48550/arxiv.1610.02136 · Cited by 1,968
RUSBoost: A Hybrid Approach to Alleviating Class Imbalance
Seiffert, C., Khoshgoftaar, T. M., Van Hulse, J., & Napolitano, A. (2010) · IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans · doi:10.1109/tsmca.2009.2029559 · Cited by 1,867
SMOTEBoost: Improving Prediction of the Minority Class in Boosting
Chawla, N. V., Lazarevic, A., Hall, L. O., & Bowyer, K. W. (2003) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-39804-2_12 · Cited by 1,668
Training cost-sensitive neural networks with methods addressing the class imbalance problem
Zhou, Z.-H., & Liu, X.-Y. (2006) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2006.17 · Cited by 1,326
Obtaining well calibrated probabilities using Bayesian binning
Pakdaman Naeini, M., Cooper, G., & Hauskrecht, M. (2015) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v29i1.9602 · Cited by 991 · cited 3×
“Calibration Metrics: We evaluate the calibration performance based on Expected Calibration Error (ECE) [50], Maximum Calibration Error (MCE) and Brier Score (BS) [51].”
MethodCost-Sensitive Learning of Deep Feature Representations From Imbalanced Data
Khan, S. H., Hayat, M., Bennamoun, M., Sohel, F. A., & Togneri, R. (2018) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2017.2732482 · Cited by 967
Applying Convolutional Neural Networks concepts to hybrid NN-HMM model for speech recognition
Abdel-Hamid, O., Mohamed, A.-R., Jiang, H., & Penn, G. (2012) · 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2012.6288864 · Cited by 876
A survey on addressing high-class imbalance in big data
Leevy, J. L., Khoshgoftaar, T. M., Bauder, R. A., & Seliya, N. (2018) · Journal of Big Data · doi:10.1186/s40537-018-0151-6 · Cited by 759
Acharya, U. R., Fujita, H., Lih, O. S., Hagiwara, Y., Tan, J. H., & Adam, M. (2017) · Information Sciences · doi:10.1016/j.ins.2017.04.012 · Cited by 752
Learning to Reweight Examples for Robust Deep Learning
Ren, M., Zeng, W., Yang, B., & Urtasun, R. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.09050 · Cited by 740
Mazurowski, M. A., Habas, P. A., Zurada, J. M., Lo, J. Y., Baker, J. A., & Tourassi, G. D. (2008) · Neural Networks · doi:10.1016/j.neunet.2007.12.031 · Cited by 737
Model Evaluation, Model Selection, and Algorithm Selection in Machine Learning
Raschka, S. (2018) · arXiv · doi:10.48550/arxiv.1811.12808 · Cited by 735 · cited 1×
“Stratification enables to randomly split the data set while retaining the correct class distribution in each subset, which is the recommended way of splitting data under class imbalance [55].”
BackgroundA methodological approach to the classification of dermoscopy images
Celebi, M. E., Kingravi, H. A., Uddin, B., Iyatomi, H., Aslandogan, Y. A., Stoecker, W. V., & Moss, R. H. (2007) · Computerized Medical Imaging and Graphics · doi:10.1016/j.compmedimag.2007.01.003 · Cited by 693
Class imbalances versus small disjuncts
Jo, T., & Japkowicz, N. (2004) · ACM SIGKDD Explorations Newsletter · doi:10.1145/1007730.1007737 · Cited by 683
Calibrating Probability with Undersampling for Unbalanced Classification
Pozzolo, A. D., Caelen, O., Johnson, R. A., & Bontempi, G. (2015) · 2015 IEEE Symposium Series on Computational Intelligence · doi:10.1109/ssci.2015.33 · Cited by 667 · cited 3×
“While classification of data featuring high class imbalance has received attention in prior research, reliability of class membership probabilities in the presence of class imbalance has been previously assessed only to a very limited extent [11], [12].”
BackgroundGradient Harmonized Single-Stage Detector
Li, B., Liu, Y., & Wang, X. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33018577 · Cited by 530
Imbalanced Deep Learning by Minority Class Incremental Rectification
Dong, Q., Gong, S., & Zhu, X. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2832629 · Cited by 369
Application of Credit Card Fraud Detection: Based on Bagging Ensemble Classifier
Zareapoor, M., & Shamsolmoali, P. (2015) · Procedia Computer Science · doi:10.1016/j.procs.2015.04.201 · Cited by 355
Combating imbalance in network intrusion datasets
Cieslak, D. A., Chawla, N. V., & Striegel, A. (2006) · 2006 IEEE International Conference on Granular Computing · doi:10.1109/grc.2006.1635905 · Cited by 286
BCN20000: Dermoscopic Lesions in the Wild
Combalia, M., Codella, N., Rotemberg, V. M., Helba, B., Vilaplana, V., Reiter, O., Carrera, C., Barreiro, A., Halpern, A. C., Puig, S., et al. (2019) · LA Referencia (Red Federada de Repositorios Institucionales de Publicaciones Científicas) · doi:10.48550/arxiv.1908.02288 · Cited by 284
A Cost-Sensitive Deep Belief Network for Imbalanced Classification
Zhang, C., Tan, K. C., Li, H., & Hong, G. S. (2019) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2018.2832648 · Cited by 282
Reliability, sufficiency, and the decomposition of proper scores
Bröcker, J. (2009) · Quarterly Journal of the Royal Meteorological Society · doi:10.1002/qj.456 · Cited by 278
Learning from Imbalanced Data Sets with Weighted Cross-Entropy Function
Aurelio, Y. S., de Almeida, G. M., de Castro, C. L., & Braga, A. P. (2019) · Neural Processing Letters · doi:10.1007/s11063-018-09977-1 · Cited by 273
Local neighbourhood extension of SMOTE for mining imbalanced data
Maciejewski, T., & Stefanowski, J. (2011) · 2011 IEEE Symposium on Computational Intelligence and Data Mining (CIDM) · doi:10.1109/cidm.2011.5949434 · Cited by 255
The Influence of Class Imbalance on Cost-Sensitive Learning: An Empirical Study
Liu, X.-Y., & Zhou, Z.-H. (2006) · Sixth International Conference on Data Mining (ICDM'06) · doi:10.1109/icdm.2006.158 · Cited by 248
Classification of ECG signals based on 1D convolution neural network
Li, D., Zhang, J., Zhang, Q., & Wei, X. (2017) · 2017 IEEE 19th International Conference on e-Health Networking, Applications and Services (Healthcom) · doi:10.1109/healthcom.2017.8210784 · Cited by 236
Novel Cost-Sensitive Approach to Improve the Multilayer Perceptron Performance on Imbalanced Data
Castro, C. L., & Braga, A. P. (2013) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2013.2246188 · Cited by 235
Kovács, B., Tinya, F., Németh, C., & Ódor, P. (2020) · Ecological Applications · doi:10.1002/eap.2043 · Cited by 233
Measuring Calibration in Deep Learning
Nixon, J., Dusenberry, M., Jerfel, G., Nguyen, T., Liu, J., Zhang, L., & Tran, D. (2019) · arXiv · doi:10.48550/arxiv.1904.01685 · Cited by 209
Improving Software-Quality Predictions With Data Sampling and Boosting
Seiffert, C., Khoshgoftaar, T. M., & Hulse, J. V. (2009) · IEEE Transactions on Systems Man and Cybernetics - Part A Systems and Humans · doi:10.1109/tsmca.2009.2027131 · Cited by 122
Wu, Z., Guo, Y., Lin, W., Yu, S., & Ji, Y. (2018) · Sensors · doi:10.3390/s18041096 · Cited by 91
Fault diagnosis based on imbalance modified kernel Fisher discriminant analysis
Zhu, Z., & Song, Z. (2010) · Chemical Engineering Research and Design · doi:10.1016/j.cherd.2010.01.005 · Cited by 89
IMPROVING ELECTRIC FRAUD DETECTION USING CLASS IMBALANCE STRATEGIES
Martino, J. M. D., Decia, F., Molinelli, J., & Fernández, A. (2012) · International Conference on Pattern Recognition Applications and Methods · doi:10.5220/0003768401350141 · Cited by 76
Class imbalance problem in the Network Intrusion Detection Systems
Rodda, S., & Erothi, U. S. R. (2016) · International Conference on Electrical, Electronics, and Optimization Techniques · doi:10.1109/iceeot.2016.7755181 · Cited by 69
Improving class probability estimates for imbalanced data
Wallace, B., & Dahabreh, I. J. (2013) · Knowledge and Information Systems · doi:10.1007/s10115-013-0670-6 · Cited by 66
Verification of forecasts expressed in terms of probability
BRIER, G. W. (1950) · Monthly Weather Review · doi:10.1175/1520-0493(1950)078%3c0001:vofeit%3e2.0.co;2
VERIFICATION OF FORECASTS EXPRESSED IN TERMS OF PROBABILITY
BRIER, G. W. (1950) · Monthly Weather Review · doi:10.1175/1520-0493(1950)078<0001:vofeit>2.0.co;2
Cited by
Yue, G., Wei, P., Zhou, T., Song, Y., Zhao, C., Wang, T., & Lei, B. (2024) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2023.3319516 · Cited by 24
Metadata
- Authors
- Fernando, K. R. M., & Tsokos, C. P.
- Venue
- IEEE Transactions on Neural Networks and Learning Systems
- Published
- 2022-07
- Volume
- 33
- Issue
- 7
- Pages
- 2940-2951
- ISSN
- 2162-237X (ISSN-L)
- 2162-2388
- OpenAlex
- W3120093105
- Cited by
- 329
- References
- 60
- FWCI
- 14.73
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.