From concept drift to model degradation: An overview on performance-aware drift detectors
Open accessThe DOI matches its Crossref record; 143 of the 143 references it cites were found in a registry or an index.
Abstract
The dynamicity of real-world systems poses a significant challenge to deployed predictive machine learning (ML) models. Changes in the system on which the ML model has been trained may lead to performance degradation during the system’s life cycle. Recent advances that study non-stationary environments have mainly focused on identifying and addressing such changes caused by a phenomenon called concept drift. Different terms have been used in the literature to refer to the same type of concept drift and the same term for various types. This lack of unified terminology is set out to create confusion on distinguishing between different concept drift variants. In this paper, we start by grouping concept drift types by their mathematical definitions and survey the different terms used in the literature to build a consolidated taxonomy of the field. We also review and classify performance-based concept drift detection methods proposed in the last decade. These methods utilize the predictive model’s performance degradation to signal substantial changes in the systems. The classification is outlined in a hierarchical diagram to provide an orderly navigation between the methods. We present a comprehensive analysis of the main attributes and strategies for tracking and evaluating the model’s performance in the predictive system. The paper concludes by discussing open research challenges and possible research directions.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Extreme learning machine: Theory and applications
Huang, G.-B., Zhu, Q.-Y., & Siew, C.-K. (2006) · Neurocomputing · doi:10.1016/j.neucom.2005.12.126 · Cited by 13,285
PAGE, E. S. (1954) · Biometrika · doi:10.1093/biomet/41.1-2.100 · Cited by 5,739
Proceedings of the 25th international conference on Machine learning
Cohen, W. W., McCallum, A., & Roweis, S. T. (2008) · Cited by 5,691
Machine learning: Algorithms, real-world applications and research directions
Sarker, I. H. (2021) · SN Computer Science · doi:10.1007/s42979-021-00592-x · Cited by 5,583
Probability Inequalities for Sums of Bounded Random Variables
Hoeffding, W. (1963) · Journal of the American Statistical Association · doi:10.1080/01621459.1963.10500830 · Cited by 4,852
A survey on concept drift adaptation
Gama, J., Žliobaitė, I., Bifet, A., Pechenizkiy, M., & Bouchachia, A. (2014) · ACM Computing Surveys · doi:10.1145/2523813 · Cited by 3,735
Sulla determinazione empirica di una legge di distribuzione
Kolmogorov, A. (1933) · Medical Entomology and Zoology · Cited by 3,597
Detection of Abrupt Changes: Theory and Application
Riedel, K. S. (1994) · Technometrics · doi:10.1080/00401706.1994.10485821 · Cited by 3,518
On the Interpretation of χ 2 from Contingency Tables, and the Calculation of P
Fisher, R. A. (1922) · Journal of the Royal Statistical Society · doi:10.2307/2340521 · Cited by 2,953
International Conference on Artificial Neural Networks
(1990) · Neural Networks · doi:10.1016/0893-6080(90)90065-s · Cited by 2,088
Pattern Classification (2nd Edition)
Duda, R. O., Hart, P. E., & Stork, D. G. (2000) · Wiley-Interscience eBooks · Cited by 2,070
The Weighted Majority Algorithm
Littlestone, N., & Warmuth, M. K. (1994) · Information and Computation · doi:10.1006/inco.1994.1009 · Cited by 1,975
A Fast and Accurate Online Sequential Learning Algorithm for Feedforward Networks
Liang, N.-Y., Huang, G.-B., Saratchandran, P., & Sundararajan, N. (2006) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2006.880583 · Cited by 1,965
Learning in the Presence of Concept Drift and Hidden Contexts
Widmer, G., & Kubat, M. (1996) · Machine Learning · doi:10.1023/a:1018046501280 · Cited by 1,857
Improving predictive inference under covariate shift by weighting the log-likelihood function
Shimodaira, H. (2000) · Journal of Statistical Planning and Inference · doi:10.1016/s0378-3758(00)00115-4 · Cited by 1,678
Nonlinear neural networks: Principles, mechanisms, and architectures
Grossberg, S. (1988) · Neural Networks · doi:10.1016/0893-6080(88)90021-4 · Cited by 1,623
Learning under Concept Drift: A Review
Lu, J., Liu, A., Dong, F., Gu, F., Gama, J., & Zhang, G. (2018) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2018.2876857 · Cited by 1,603
On the method of bounded differences
McDiarmid, C. (1989) · Surveys in Combinatorics, 1989 · doi:10.1017/cbo9781107359949.008 · Cited by 1,425
Mining with rarity: a unifying framework
Weiss, G. M. (2004) · ACM SIGKDD Explorations Newsletter · doi:10.1145/1007730.1007734 · Cited by 1,393
Mining concept-drifting data streams using ensemble classifiers
Wang, H., Fan, W., Yu, P. S., & Han, J. (2003) · Proceedings of the ninth ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/956750.956778 · Cited by 1,256
A streaming ensemble algorithm (SEA) for large-scale classification
Street, W. N., & Kim, Y. (2001) · Proceedings of the seventh ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/502512.502568 · Cited by 1,205
Ensemble learning for data stream analysis: A survey
Krawczyk, B., Minku, L. L., Gama, J., Stefanowski, J., & Woźniak, M. (2017) · Information Fusion · doi:10.1016/j.inffus.2017.02.004 · Cited by 1,050
A unifying view on dataset shift in classification
Moreno-Torres, J. G., Raeder, T., Alaiz-Rodríguez, R., Chawla, N. V., & Herrera, F. (2012) · Pattern Recognition · doi:10.1016/j.patcog.2011.06.019 · Cited by 1,029
2014 IEEE International Conference on Data Mining
(2014) · doi:10.1109/icdm33987.2014 · Cited by 1,007
Incremental Learning of Concept Drift in Nonstationary Environments
Elwell, R., & Polikar, R. (2011) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2011.2160459 · Cited by 931
The problem of concept drift: definitions and related work
Tsymbal, A. (2004) · Cited by 929
Learn++: an incremental learning algorithm for supervised neural networks
Polikar, R., Upda, L., Upda, S. S., & Honavar, V. (2001) · IEEE Transactions on Systems, Man and Cybernetics, Part C (Applications and Reviews) · doi:10.1109/5326.983933 · Cited by 888
Deep Learning: A Critical Appraisal
Marcus, G. (2018) · arXiv · doi:10.48550/arxiv.1801.00631 · Cited by 853
Learning in Nonstationary Environments: A Survey
Ditzler, G., Roveri, M., Alippi, C., & Polikar, R. (2015) · IEEE Computational Intelligence Magazine · doi:10.1109/mci.2015.2471196 · Cited by 793
2018 International Joint Conference on Neural Networks (IJCNN)
(2018) · doi:10.1109/ijcnn40505.2018 · Cited by 746
Breakthroughs in Statistics: Foundations and Basic Theory
(1992) · Springer Series in Statistics · doi:10.1007/978-1-4612-0919-5 · Cited by 714
Dataset Shift in Machine Learning
Adams, N. (2010) · Journal of the Royal Statistical Society Series A: Statistics in Society · doi:10.1111/j.1467-985x.2009.00624_10.x · Cited by 694
2014 International Joint Conference on Neural Networks (IJCNN)
(2014) · doi:10.1109/ijcnn32435.2014 · Cited by 685
Dynamic Weighted Majority: An Ensemble Method for Drifting Concepts
Kolter, J. Z., & Maloof, M. A. (2007) · doi:10.5555/1314498.1390333 · Cited by 648
Simple Explanation of the No-Free-Lunch Theorem and Its Implications
Ho, Y. C., & Pepyne, D. L. (2002) · Journal of Optimization Theory and Applications · doi:10.1023/a:1021251113462 · Cited by 562
2011 International Joint Conference on Neural Networks
(2010) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2010.2051284 · Cited by 559
Webb, G. I., Hyde, R., Cao, H., Nguyen, H. L., & Petitjean, F. (2016) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-015-0448-4 · Cited by 536
On evaluating stream learning algorithms
Gama, J., Sebastião, R., & Rodrigues, P. P. (2013) · Machine Learning · doi:10.1007/s10994-012-5320-9 · Cited by 532
A survey on data preprocessing for data stream mining: Current status and future directions
Ramírez-Gallego, S., Krawczyk, B., García, S., Woźniak, M., & Herrera, F. (2017) · Neurocomputing · doi:10.1016/j.neucom.2017.01.078 · Cited by 502
Online and Non-Parametric Drift Detection Methods Based on Hoeffding’s Bounds
Frias-Blanco, I., Campo-Avila, J. D., Ramos-Jimenez, G., Morales-Bueno, R., Ortiz-Diaz, A., & Caballero-Mota, Y. (2015) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2014.2345382 · Cited by 469
The Impact of Diversity on Online Ensemble Learning in the Presence of Concept Drift
Minku, L. L., White, A. P., & Yao, X. (2010) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2009.156 · Cited by 440
Learning drifting concepts: Example selection vs. example weighting
Klinkenberg, R. (2004) · Intelligent Data Analysis · doi:10.3233/ida-2004-8305 · Cited by 438
DDD: A New Ensemble Approach for Dealing with Concept Drift
Minku, L. L., & Yao, X. (2012) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2011.58 · Cited by 431
Reacting to Different Types of Concept Drift: The Accuracy Updated Ensemble Algorithm
Brzezinski, D., & Stefanowski, J. (2014) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2013.2251352 · Cited by 423
2020 IEEE Symposium on Security and Privacy (SP)
(2020) · doi:10.1109/sp40000.2020 · Cited by 423
Incremental Learning of Concept Drift from Streaming Imbalanced Data
Ditzler, G., & Polikar, R. (2013) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2012.136 · Cited by 406
Exponentially weighted moving average charts for detecting concept drift
Ross, G. J., Adams, N. M., Tasoulis, D. K., & Hand, D. J. (2012) · Pattern Recognition Letters · doi:10.1016/j.patrec.2011.08.019 · Cited by 404
Analysis of Explainers of Black Box Deep Neural Networks for Computer Vision: A Survey
Buhrmester, V., Münch, D., & Arens, M. (2021) · Machine Learning and Knowledge Extraction · doi:10.3390/make3040048 · Cited by 403
2015 International Joint Conference on Neural Networks (IJCNN)
(2015) · doi:10.1109/ijcnn34689.2015 · Cited by 397
Proceedings of the 2007 SIAM International Conference on Data Mining
(2007) · doi:10.1137/1.9781611972771 · Cited by 380
An Overview of Concept Drift Applications
Žliobaitė, I., Pechenizkiy, M., & Gama, J. (2016) · Studies in Big Data · doi:10.1007/978-3-319-26989-4_4 · Cited by 379
A Systematic Study of Online Class Imbalance Learning With Concept Drift
Wang, S., Minku, L. L., & Yao, X. (2018) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2017.2771290 · Cited by 331
Bai, X., Wang, X., Liu, X., Liu, Q., Song, J., Sebe, N., & Kim, B. (2021) · Pattern Recognition · doi:10.1016/j.patcog.2021.108102 · Cited by 313
Proceedings of the 2020 SIAM International Conference on Data Mining
(2020) · Society for Industrial and Applied Mathematics eBooks · doi:10.1137/1.9781611976236 · Cited by 312
Spiking Neural Networks and online learning: An overview and perspectives
Lobo, J. L., Del Ser, J., Bifet, A., & Kasabov, N. (2020) · Neural Networks · doi:10.1016/j.neunet.2019.09.004 · Cited by 304
Learning from streaming data with concept drift and imbalance: an overview
Hoens, T. R., Polikar, R., & Chawla, N. V. (2012) · Progress in Artificial Intelligence · doi:10.1007/s13748-011-0008-0 · Cited by 301
Concept whitening for interpretable image recognition
Chen, Z., Bei, Y., & Rudin, C. (2020) · Nature Machine Intelligence · doi:10.1038/s42256-020-00265-z · Cited by 279
The 2012 International Joint Conference on Neural Networks (IJCNN)
(2012) · doi:10.1109/ijcnn20752.2012 · Cited by 269
Machine Learning in Non-Stationary Environments: Introduction to Covariate Shift Adaptation
Sugiyama, M., & Kawanabe, M. (2012) · Cited by 263
Learning under Concept Drift: an Overview
Žliobaitė, I. (2010) · arXiv (Cornell University) · doi:10.48550/arxiv.1010.4784 · Cited by 228
Discussion and review on evolving data streams and concept drift adapting
Khamassi, I., Sayed-Mouchaweh, M., Hammami, M., & Ghédira, K. (2018) · Evolving Systems · doi:10.1007/s12530-016-9168-2 · Cited by 223
Tracking recurring contexts using ensemble classifiers: an application to email filtering
Katakis, I., Tsoumakas, G., & Vlahavas, I. (2010) · Knowledge and Information Systems · doi:10.1007/s10115-009-0206-2 · Cited by 213
2021 International Joint Conference on Neural Networks (IJCNN)
(2021) · doi:10.1109/ijcnn52387.2021 · Cited by 213
RDDM: Reactive drift detection method
Barros, R. S. M., Cabral, D. R. L., Gonçalves, P. M., & Santos, S. G. T. C. (2017) · Expert Systems with Applications · doi:10.1016/j.eswa.2017.08.023 · Cited by 209
A case-based technique for tracking concept drift in spam filtering
Delany, S. J., Cunningham, P., Tsymbal, A., & Coyle, L. (2005) · Knowledge-Based Systems · doi:10.1016/j.knosys.2004.10.002 · Cited by 206
An overview of unsupervised drift detection methods
Gemaque, R. N., Costa, A. F. J., Giusti, R., & Santos, E. M. D. (2020) · Wiley Interdisciplinary Reviews Data Mining and Knowledge Discovery · doi:10.1002/widm.1381 · Cited by 203
Drift-Aware Methodology for Anomaly Detection in Smart Grid
Fenza, G., Gallo, M., & Loia, V. (2019) · IEEE Access · doi:10.1109/access.2019.2891315 · Cited by 199
Concept drift detection via competence models
Lu, N., Zhang, G., & Lu, J. (2014) · Artificial Intelligence · doi:10.1016/j.artint.2014.01.001 · Cited by 198
Dynamic integration of classifiers for handling concept drift
Tsymbal, A., Pechenizkiy, M., Cunningham, P., & Puuronen, S. (2008) · Information Fusion · doi:10.1016/j.inffus.2006.11.002 · Cited by 195
Adaptive concept drift detection
Dries, A., & Rückert, U. (2009) · Statistical Analysis and Data Mining The ASA Data Science Journal · doi:10.1002/sam.10054 · Cited by 191
2012 IEEE 12th International Conference on Data Mining Workshops
(2012) · doi:10.1109/icdmw30884.2012 · Cited by 181
Combining block-based and online methods in learning ensembles from concept drifting data streams
Brzezinski, D., & Stefanowski, J. (2014) · Information Sciences · doi:10.1016/j.ins.2013.12.011 · Cited by 179
An Overview on Concept Drift Learning
Iwashita, A. S., & Papa, J. P. (2019) · IEEE Access · doi:10.1109/access.2018.2886026 · Cited by 179
Ensemble of subset online sequential extreme learning machine for class imbalance and concept drift
Mirza, B., Lin, Z., & Liu, N. (2014) · Neurocomputing · doi:10.1016/j.neucom.2014.03.075 · Cited by 167
Data stream mining: methods and challenges for handling concept drift
Wares, S., Isaacs, J. P., & Elyan, E. (2019) · SN Applied Sciences · doi:10.1007/s42452-019-1433-0 · Cited by 167
A large-scale comparison of concept drift detectors
Barros, R. S. M., & Santos, S. G. T. C. (2018) · Information Sciences · doi:10.1016/j.ins.2018.04.014 · Cited by 164
Accuracy Updated Ensemble for Data Streams with Concept Drift
Brzeziński, D., & Stefanowski, J. (2011) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-21222-2_19 · Cited by 161
Dynamic extreme learning machine for data stream classification
Xu, S., & Wang, J. (2017) · Neurocomputing · doi:10.1016/j.neucom.2016.12.078 · Cited by 136
A comprehensive active learning method for multiclass imbalanced data streams with concept drift
Liu, W., Zhang, H., Ding, Z., Liu, Q., & Zhu, C. (2021) · Knowledge-Based Systems · doi:10.1016/j.knosys.2021.106778 · Cited by 134
Wilcoxon Rank Sum Test Drift Detector
de Barros, R. S. M., Hidalgo, J. I. G., & Cabral, D. R. D. L. (2017) · Neurocomputing · doi:10.1016/j.neucom.2017.10.051 · Cited by 133
Knowledge-Preserving Incremental Social Event Detection via Heterogeneous GNNs
Cao, Y., Peng, H. M., Wu, J. X., Dou, Y., Li, J., & Yu, P. S. (2021) · International World Wide Web Conference (WWW) · doi:10.1145/3442381.3449834 · Cited by 132
Sixth International Conference on Data Mining (ICDM'06)
(2006) · doi:10.1109/icdm11877.2006 · Cited by 131
Learning Concept Drift with a Committee of Decision Trees
Stanley, K. O. (2003) · Cited by 127
Using multiple windows to track concept drift
Lazarescu, M., Venkatesh, S., & Bui, H. (2004) · Intelligent Data Analysis · doi:10.3233/ida-2004-8103 · Cited by 127
RCD: A recurring concept drift framework
Gonçalves, P. M., Jr., & de Barros, R. S. M. (2013) · Pattern Recognition Letters · doi:10.1016/j.patrec.2013.02.005 · Cited by 126
Detecting concept drift in data streams using model explanation
Demšar, J., & Bosnić, Z. (2017) · Expert Systems with Applications · doi:10.1016/j.eswa.2017.10.003 · Cited by 124
Applying lazy learning algorithms to tackle concept drift in spam filtering
Fdez‐Riverola, F., Iglesias, E. L., Díaz, F., Méndez, J. R., & Corchado, J. M. (2006) · Expert Systems with Applications · doi:10.1016/j.eswa.2006.04.011 · Cited by 118
A Novel Concept Drift Detection Method for Incremental Learning in Nonstationary Environments
Yang, Z., Al‐Dahidi, S., Baraldi, P., Zio, E., & Montelatici, L. (2019) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2019.2900956 · Cited by 118
Data-driven decision support under concept drift in streamed big data
Lü, J., Liu, A., Song, Y., & Zhang, G. (2019) · Complex & Intelligent Systems · doi:10.1007/s40747-019-00124-4 · Cited by 116
2016 IEEE 32nd International Conference on Data Engineering (ICDE)
(2016) · doi:10.1109/icde37718.2016 · Cited by 116
Concept drift detection based on Fisher’s Exact test
Cabral, D. R. D. L., & de Barros, R. S. M. (2018) · Information Sciences · doi:10.1016/j.ins.2018.02.054 · Cited by 113
Analyzing concept drift and shift from sample data
Webb, G. I., Lee, L. K., Goethals, B., & Petitjean, F. (2018) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-018-0554-1 · Cited by 110
No Free Lunch Theorem for concept drift detection in streaming data classification: A review
Hu, H., Kantardzic, M. M., & Sethi, T. S. (2019) · Wiley Interdisciplinary Reviews Data Mining and Knowledge Discovery · doi:10.1002/widm.1327 · Cited by 110
2021 IEEE Globecom Workshops (GC Wkshps)
(2021) · doi:10.1109/gcwkshps52748.2021 · Cited by 110
Tolerating Concept and Sampling Shift in Lazy Learning Using Prediction Error Context Switching
Salganicoff, M. (1997) · Artificial Intelligence Review · doi:10.1023/a:1006515405170 · Cited by 102
Pesaranghader, A., Viktor, H. L., & Paquet, E. (2018) · Machine Learning · doi:10.1007/s10994-018-5719-z · Cited by 101
Survey of distance measures for quantifying concept drift and shift in numeric data
Goldenberg, I., & Webb, G. I. (2018) · Knowledge and Information Systems · doi:10.1007/s10115-018-1257-z · Cited by 98
An overview and comprehensive comparison of ensembles for concept drift
de Barros, R. S. M., & Santos, S. G. T. D. C. (2019) · Information Fusion · doi:10.1016/j.inffus.2019.03.006 · Cited by 98
Mirza, B., & Lin, Z. (2016) · Neural Networks · doi:10.1016/j.neunet.2016.04.008 · Cited by 96
Ensuring cybersecurity of smart grid against data integrity attacks under concept drift
Mohammadpourfard, M., Weng, Y., Pechenizkiy, M., Tajdinian, M., & Mohammadi‐Ivatloo, B. (2020) · International Journal of Electrical Power & Energy Systems · doi:10.1016/j.ijepes.2020.105947 · Cited by 94
Xu, R., Cheng, Y., Liu, Z., Xie, Y., & Yang, Y. J. (2020) · Future Generation Computer Systems · doi:10.1016/j.future.2020.05.035 · Cited by 88
Lughofer, E. D., Weigl, E., Heidl, W., Eitzinger, C., & Radauer, T. (2016) · Information Sciences · doi:10.1016/j.ins.2016.03.034 · Cited by 87
Handling adversarial concept drift in streaming data
Sethi, T. S., & Kantardzic, M. M. (2017) · Expert Systems with Applications · doi:10.1016/j.eswa.2017.12.022 · Cited by 83
Proceedings of the Thirtieth international conference on Very large data bases - Volume 30
Nascimento, M. A., ÖZSU, M. T., Kossmann, D., Miller, R. J., Blakeley, J. A., & Schiefer, K. B. (2004) · Cited by 81
An on-line weighted ensemble of regressor models to handle concept drifts
Alcalá, S. G. S., & Araújo, R. A. M. (2014) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2014.10.003 · Cited by 75
Concept drift detection and adaptation with hierarchical hypothesis testing
Yu, S., Abraham, Z., Wang, H., Shah, M., Wei, Y., & Príncipe, J. C. (2019) · Journal of the Franklin Institute · doi:10.1016/j.jfranklin.2019.01.043 · Cited by 72
Concept Drift and Anomaly Detection in Graph Streams
Zambon, D., Alippi, C., & Livi, L. (2018) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2018.2804443 · Cited by 68
Detecting concept drift: An information entropy based method using an adaptive sliding window
Du, L., Song, Q., & Jia, X. (2014) · Intelligent Data Analysis · doi:10.3233/ida-140645 · Cited by 66
Machine Learning and Knowledge Discovery in Databases
Calders, T., & Nancy, E. P. 2. (2014) · Lecture notes in computer science · doi:10.1007/978-3-662-44851-9 · Cited by 64
A heterogeneous online learning ensemble for non-stationary environments
Idrees, M. M., Minku, L. L., Stahl, F. T., & Badii, A. (2019) · Knowledge-Based Systems · doi:10.1016/j.knosys.2019.104983 · Cited by 57
Accurate detecting concept drift in evolving data streams
Yan, M. M. W. (2020) · ICT Express · doi:10.1016/j.icte.2020.05.011 · Cited by 57
A Selective Detector Ensemble for Concept Drift Detection
Du, L., Song, Q., Zhu, L., & Zhu, X. (2014) · The Computer Journal · doi:10.1093/comjnl/bxu050 · Cited by 54
Don’t Pay for Validation: Detecting Drifts from Unlabeled data Using Margin Density
Sethi, T. S., & Kantardzic, M. M. (2015) · Procedia Computer Science · doi:10.1016/j.procs.2015.07.284 · Cited by 53
A drift detection method based on dynamic classifier selection
Pinage, F. A., Santos, E. M. D., & da Gama, J. M. P. (2019) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-019-00656-w · Cited by 53
Analysis of Explainers of Black Box Deep Neural Networks for Computer Vision: A SurveyPreprint
Buhrmester, V., Münch, D., & Arens, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.12116 · Cited by 52
Adaptive online incremental learning for evolving data streams
Zhang, S., Liu, J., & Zuo, X. (2021) · Applied Soft Computing · doi:10.1016/j.asoc.2021.107255 · Cited by 51
A grid density based framework for classifying streaming data in the presence of concept drift
Sethi, T. S., Kantardzic, M. M., & Hu, H. (2015) · Journal of Intelligent Information Systems · doi:10.1007/s10844-015-0358-3 · Cited by 47
A Segment-Based Drift Adaptation Method for Data Streams
Song, Y., Lü, J., Liu, A., Lu, H., & Zhang, G. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3062062 · Cited by 47
Recurring concept meta-learning for evolving data streams
Anderson, R., Koh, Y. S., Dobbie, G., & Bifet, A. (2019) · Expert Systems with Applications · doi:10.1016/j.eswa.2019.112832 · Cited by 45
Learning Data Streams With Changing Distributions and Temporal Dependency
Song, Y., Lü, J., Lu, H., & Zhang, G. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3122531 · Cited by 45
Self-Adaptive Windowing Approach for Handling Complex Concept Drift
Khamassi, I., Sayed‐Mouchaweh, M., Hammami, M., & Ghédira, K. (2015) · Cognitive Computation · doi:10.1007/s12559-015-9341-0 · Cited by 42
2021 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE)
(2021) · doi:10.1109/fuzz45933.2021 · Cited by 39
LAK22: 12th International Learning Analytics and Knowledge Conference
Wise, A. F., Martinez-Maldonado, R., & Hilliger, I. (2022) · doi:10.1145/3506860 · Cited by 36
Wang, K., Lü, J., Liu, A., Zhang, G., & Xiong, L. (2021) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2021.3109796 · Cited by 35
Drift mining in data: A framework for addressing drift in classification
Hofer, V., & Krempl, G. (2012) · Computational Statistics & Data Analysis · doi:10.1016/j.csda.2012.07.007 · Cited by 34
Detecting virtual concept drift of regressors without ground truth values
Oikarinen, E., Tiittanen, H., Henelius, A., & Puolamäki, K. (2021) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-021-00739-7 · Cited by 32
Predictive Handling of Asynchronous Concept Drifts in Distributed Environments
Ang, H. H., Gopalkrishnan, V., Žliobaitė, I., Pechenizkiy, M., & Hoi, S. C. H. (2012) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2012.172 · Cited by 31
Incremental Market Behavior Classification in Presence of Recurring Concepts
Suárez‐Cetrulo, A. L., Cervantes, A., & Quintana, D. (2019) · Entropy · doi:10.3390/e21010025 · Cited by 27
An Approach For Concept Drift Detection in a Graph Stream Using Discriminative Subgraphs
Paudel, R., & Eberle, W. (2020) · ACM Transactions on Knowledge Discovery from Data · doi:10.1145/3406243 · Cited by 24
Khezri, S., Tanha, J., Ahmadi, A., & Sharifi, A. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.02.031 · Cited by 24
Meta-cognitive Recurrent Recursive Kernel OS-ELM for concept drift handling
Liu, Z., Loo, C. K., & Seera, M. (2018) · Applied Soft Computing · doi:10.1016/j.asoc.2018.11.006 · Cited by 22
Sidhu, P., & Bhatia, M. P. S. (2015) · International Journal of Machine Learning and Cybernetics · doi:10.1007/s13042-015-0366-1 · Cited by 20
A two ensemble system to handle concept drifting data streams: recurring dynamic weighted majority
Sidhu, P., & Bhatia, M. P. S. (2017) · International Journal of Machine Learning and Cybernetics · doi:10.1007/s13042-017-0738-9 · Cited by 20
Using spectral entropy and bernoulli map to handle concept drift
Chikushi, R. T. M., de Barros, R. S. M., da Silva, M. G. N. M., & Maciel, B. I. F. (2020) · Expert Systems with Applications · doi:10.1016/j.eswa.2020.114114 · Cited by 20
Concept drift mining of portfolio selection factors in stock market
Hu, Y. Q., Liu, K., Zhang, X., Xie, K., Chen, W., Zeng, Y., & Liu, M. (2015) · Electronic Commerce Research and Applications · doi:10.1016/j.elerap.2015.06.002 · Cited by 18
Ensembles of Heterogeneous Concept Drift Detectors - Experimental Study
Woźniak, M., Ksieniewicz, P., Cyganek, B., & Walkowiak, K. M. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-45378-1_48 · Cited by 17
Towards a real-time unsupervised estimation of predictive model degradation
Cerquitelli, T., Proto, S., Ventura, F., Apiletti, D., & Baralis, E. (2019) · doi:10.1145/3350489.3350494 · Cited by 17
Mahdi, O. A., Pardede, E., & Ali, N. (2021) · Cluster Computing · doi:10.1007/s10586-021-03267-7 · Cited by 13
Long short-term memory self-adapting online random forests for evolving data stream regression
Zhong, Y., Yang, H., Zhang, Y., Li, P., & Ren, C. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.05.026 · Cited by 11
LUNAR: Cellular automata for drifting data streams
Lobo, J. L., Ser, J. D., & Herrera, F. (2020) · Information Sciences · doi:10.1016/j.ins.2020.08.064 · Cited by 6
Cited by
Data drift in medical machine learning: implications and potential remedies
Sahiner, B., Chen, W., Samala, R. K., & Petrick, N. (2023) · The British Journal of Radiology · doi:10.1259/bjr.20220878 · Cited by 279
Towards Trustworthy Machine Learning in Production: An Overview of the Robustness in MLOps Approach
Bayram, F., & Ahmed, B. S. (2025) · ACM Computing Surveys · doi:10.1145/3708497 · Cited by 48
Self-citationIs your anomaly detector ready for change? Adapting aiops solutions to the real world
Poenaru-Olaru, L., Karpova, N., Cruz, L., Rellermeyer, J. S., & van Deursen, A. (2024) · Proceedings of the IEEE/ACM 3rd International Conference on AI Engineering - Software Engineering for AI · doi:10.1145/3644815.3644961 · Cited by 7
Metadata
- Authors
- Bayram, F., Ahmed, B. S., & Kassler, A.
- Venue
- Knowledge-Based Systems
- Publisher
- Elsevier BV
- Published
- 2022-06
- Volume
- 245
- Pages
- 108632
- ISSN
- 0950-7051 (ISSN-L)
- OpenAlex
- W4220851938
- Cited by
- 407
- References
- 143
- FWCI
- 23.83
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.