Mining concept-drifting data streams using ensemble classifiers
The DOI matches its Crossref record; 23 of the 23 references it cites were found in a registry or an index.
Abstract
Recently, mining data streams with concept drifts for actionable insights has become an important and challenging task for a wide range of applications including credit card fraud protection, target marketing, network intrusion detection, etc. Conventional knowledge discovery tools are facing two challenges, the overwhelming volume of the streaming data, and the concept drifts. In this paper, we propose a general framework for mining concept-drifting data streams using weighted ensemble classifiers. We train an ensemble of classification models, such as C4.5, RIPPER, naive Beyesian, etc., from sequential chunks of the data stream. The classifiers in the ensemble are judiciously weighted based on their expected classification accuracy on the test data under the time-evolving environment. Thus, the ensemble approach improves both the efficiency in learning the model and the accuracy in performing classification. Our empirical study shows that the proposed methods have substantial advantage over single-classifier approaches in prediction accuracy, and the ensemble framework is effective for a variety of classification models.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
(1992) · Cited by 23,431
Cohen, W. W. (1995) · Machine Learning Proceedings 1995 · doi:10.1016/b978-1-55860-377-6.50023-2 · Cited by 3,787
Neural Networks and the Bias/Variance Dilemma
Geman, S., Bienenstock, E., & Doursat, R. (1992) · Neural Computation · doi:10.1162/neco.1992.4.1.1 · Cited by 3,578 · cited 1×
“In addition to the Bayes error, the remaining error of the classifier can be decomposed into two parts, bias and variance [17, 22, 8].”
BackgroundAn Empirical Comparison of Voting Classification Algorithms: Bagging, Boosting, and Variants
Bauer, E., & Kohavi, R. (1999) · Machine Learning · doi:10.1023/a:1007515423169 · Cited by 2,638
Models and issues in data stream systems
Babcock, B., Babu, S., Datar, M., Motwani, R., & Widom, J. (2002) · Proceedings of the twenty-first ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems · doi:10.1145/543613.543615 · Cited by 2,495
Mining high-speed data streams
Domingos, P., & Hulten, G. (2000) · Proceedings of the sixth ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/347090.347107 · Cited by 2,185
Mining time-changing data streams
Hulten, G., Spencer, L., & Domingos, P. (2001) · Proceedings of the seventh ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/502512.502529 · Cited by 1,748
A streaming ensemble algorithm (SEA) for large-scale classification
Street, W. N., & Kim, Y. (2001) · Proceedings of the seventh ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/502512.502568 · Cited by 1,205
Continuous queries over data streams
Babu, S., & Widom, J. (2001) · ACM SIGMOD Record · doi:10.1145/603867.603884 · Cited by 809
Incremental Induction of Decision Trees
Utgoff, P. E. (1989) · Machine Learning · doi:10.1023/a:1022699900025 · Cited by 791
Cited by
A survey on concept drift adaptation
Gama, J., Žliobaitė, I., Bifet, A., Pechenizkiy, M., & Bouchachia, A. (2014) · ACM Computing Surveys · doi:10.1145/2523813 · Cited by 3,735
Learning under Concept Drift: A Review
Lu, J., Liu, A., Dong, F., Gu, F., Gama, J., & Zhang, G. (2018) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2018.2876857 · Cited by 1,603
He, R., & McAuley, J. (2016) · Proceedings of the 25th International Conference on World Wide Web · doi:10.1145/2872427.2883037 · Cited by 1,459
Collaborative filtering with temporal dynamics
Koren, Y. (2009) · Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1557019.1557072 · Cited by 1,203
Ensemble learning for data stream analysis: A survey
Krawczyk, B., Minku, L. L., Gama, J., Stefanowski, J., & Woźniak, M. (2017) · Information Fusion · doi:10.1016/j.inffus.2017.02.004 · Cited by 1,050
Incremental Learning of Concept Drift in Nonstationary Environments
Elwell, R., & Polikar, R. (2011) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2011.2160459 · Cited by 931
A survey of multiple classifier systems as hybrid systems
Woźniak, M., Graña, M., & Corchado, E. (2014) · Information Fusion · doi:10.1016/j.inffus.2013.04.006 · Cited by 912
Gaber, M. M., Zaslavsky, A., & Krishnaswamy, S. (2005) · ACM SIGMOD Record · doi:10.1145/1083784.1083789 · Cited by 886
Collaborative filtering with temporal dynamics
Koren, Y. (2010) · Communications of the ACM · doi:10.1145/1721654.1721677 · Cited by 776
Online learning: A comprehensive survey
Hoi, S. C. H., Sahoo, D., Lu, J., & Zhao, P. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.04.112 · Cited by 746
Chemical gas sensor drift compensation using classifier ensembles
Vergara, A., Vembu, S., Ayhan, T., Ryan, M. A., Homer, M. L., & Huerta, R. (2012) · Sensors and Actuators B: Chemical · doi:10.1016/j.snb.2012.01.074 · Cited by 724
Predicting Taxi–Passenger Demand Using Streaming Data
Moreira-Matias, L., Gama, J., Ferreira, M., Mendes-Moreira, J., & Damas, L. (2013) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2013.2262376 · Cited by 721
Fraud detection system: A survey
Abdallah, A., Maarof, M. A., & Zainal, A. (2016) · Journal of Network and Computer Applications · doi:10.1016/j.jnca.2016.04.007 · Cited by 708
Cross-domain video concept detection using adaptive svms
Yang, J., Yan, R., & Hauptmann, A. G. (2007) · Proceedings of the 15th ACM international conference on Multimedia · doi:10.1145/1291233.1291276 · Cited by 667
Ensemble approaches for regression: A survey
Mendes-Moreira, J., Soares, C., Jorge, A. M., & Sousa, J. F. D. (2012) · ACM Computing Surveys · doi:10.1145/2379776.2379786 · Cited by 667
Dynamic Weighted Majority: An Ensemble Method for Drifting Concepts
Kolter, J. Z., & Maloof, M. A. (2007) · doi:10.5555/1314498.1390333 · Cited by 648
A Survey on Ensemble Learning for Data Stream Classification
Gomes, H. M., Barddal, J. P., Enembreck, F., & Bifet, A. (2018) · ACM Computing Surveys · doi:10.1145/3054925 · Cited by 579
“…2010], such that very often drift adaptation is achieved by assigning different weights to each of the ensemble members [Street and Kim 2001; Polikar et al. 2001; Wang et al. 2003; Kolter et al. 2003; Kolter and Maloof 2005; Brzezi´nski and Stefanowski 2011; Zhang et al. 2011b; Deckert 2011].”
BackgroundDetecting and Tracking Political Abuse in Social Media
Ratkiewicz, J., Conover, M., Meiss, M., Goncalves, B., Flammini, A., & Menczer, F. (2021) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v5i1.14127 · Cited by 579
Measuring Catastrophic Forgetting in Neural Networks
Kemker, R., McClure, M., Abitino, A., Hayes, T., & Kanan, C. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11651 · Cited by 672
Ratkiewicz (2011) · doi:10.1145/1963192.1963301 · Cited by 541
“While there exist many sophisticated statistical techniques for modeling the underlying topics present in bodies of text [9, 14], the small size of each tweet and the contextual drift present in streaming data create significant complications [50].”
BackgroundWebb, G. I., Hyde, R., Cao, H., Nguyen, H. L., & Petitjean, F. (2016) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-015-0448-4 · Cited by 536
Learned lessons in credit card fraud detection from a practitioner perspective
Dal Pozzolo, A., Caelen, O., Le Borgne, Y.-A., Waterschoot, S., & Bontempi, G. (2014) · Expert Systems with Applications · doi:10.1016/j.eswa.2014.02.026 · Cited by 530
Data stream clustering: A survey
Silva, J. A., Faria, E. R., Barros, R. C., Hruschka, E. R., Carvalho, A. C. P. L. F. D., & Gama, J. (2013) · ACM Computing Surveys · doi:10.1145/2522968.2522981 · Cited by 508
Identifying the signs of fraudulent accounts using data mining techniques
Li, S.-H., Yen, D. C., Lu, W.-H., & Wang, C. (2012) · Computers in Human Behavior · doi:10.1016/j.chb.2012.01.002 · Cited by 505
Online and Non-Parametric Drift Detection Methods Based on Hoeffding’s Bounds
Frias-Blanco, I., Campo-Avila, J. D., Ramos-Jimenez, G., Morales-Bueno, R., Ortiz-Diaz, A., & Caballero-Mota, Y. (2015) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2014.2345382 · Cited by 469
Time weight collaborative filtering
Ding, Y., & Li, X. (2005) · Proceedings of the 14th ACM international conference on Information and knowledge management · doi:10.1145/1099554.1099689 · Cited by 454
The Impact of Diversity on Online Ensemble Learning in the Presence of Concept Drift
Minku, L. L., White, A. P., & Yao, X. (2010) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2009.156 · Cited by 440
DDD: A New Ensemble Approach for Dealing with Concept Drift
Minku, L. L., & Yao, X. (2012) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2011.58 · Cited by 431
Reacting to Different Types of Concept Drift: The Accuracy Updated Ensemble Algorithm
Brzezinski, D., & Stefanowski, J. (2014) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2013.2251352 · Cited by 423
From concept drift to model degradation: An overview on performance-aware drift detectors
Bayram, F., Ahmed, B. S., & Kassler, A. (2022) · Knowledge-Based Systems · doi:10.1016/j.knosys.2022.108632 · Cited by 407
Abbasi, A., & Chen, H. (2008) · ACM Transactions on Information Systems · doi:10.1145/1344411.1344413 · Cited by 405
Classification and Novel Class Detection in Concept-Drifting Data Streams under Time Constraints
Masud, M., Gao, J., Khan, L., Han, J., & Thuraisingham, B. M. (2011) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2010.61 · Cited by 400
Self-citationClassifier Ensembles for Changing Environments
Kuncheva, L. I. (2004) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-25966-4_1 · Cited by 335
Knowledge transfer via multiple model local structure mapping
Gao, J., Fan, W., Jiang, J., & Han, J. (2008) · Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1401890.1401928 · Cited by 330
Self-citationA survey on data stream clustering and classification
Nguyen, H.-L., Woon, Y.-K., & Ng, W.-K. (2015) · Knowledge and Information Systems · doi:10.1007/s10115-014-0808-1 · Cited by 321
Learning from streaming data with concept drift and imbalance: an overview
Hoens, T. R., Polikar, R., & Chawla, N. V. (2012) · Progress in Artificial Intelligence · doi:10.1007/s13748-011-0008-0 · Cited by 301
Research issues in data stream association rule mining
Jiang, N., & Gruenwald, L. (2006) · ACM SIGMOD Record · doi:10.1145/1121995.1121998 · Cited by 287
Detecting Concept Drift Using Statistical Testing
Nishida, K., & Yamauchi, K. (2007) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-75488-6_27 · Cited by 285
Ensemble Pruning Via Semi-definite Programming
Zhang, Y., Samuel Burer, A., & Street, W. N. (2006) · doi:10.5555/1248547.1248595 · Cited by 258
Incremental tensor analysis: Theory and applications
Sun, J., Tao, D., Papadimitriou, S., Yu, P. S., & Faloutsos, C. (2008) · ACM Transactions on Knowledge Discovery from Data · doi:10.1145/1409620.1409621 · Cited by 254
Self-citationA General Framework for Mining Concept-Drifting Data Streams with Skewed Distributions
Gao, J., Fan, W., Han, J., & Yu, P. S. (2007) · Proceedings of the 2007 SIAM International Conference on Data Mining · doi:10.1137/1.9781611972771.1 · Cited by 242
“In [14] , a hyperplane is used to characterize the boundary between two classes, which is defined as a function of x i and a i .”
BackgroundMethodInfluentialSelf-citationSystematic data selection to mine concept-drifting data streams
Fan, W. (2004) · Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1014052.1014069 · Cited by 242
“Both the theorem and empirical analysis of [Wang et al., 2003] conclude that when there is concept drift, from the “same” data chunks (new and old), the weighted ensemble is more accurate than a single classifier trained from exactly the same amount of data.”
BackgroundMethodInfluentialSelf-citationConcept drift detection via competence models
Lu, N., Zhang, G., & Lu, J. (2014) · Artificial Intelligence · doi:10.1016/j.artint.2014.01.001 · Cited by 198
“These include rule-based learning [4, 8, 12], decision trees and their incremental versions [5, 13, 14], info-fuzzy networks [15], clustering [16], support vector machines [17], and case-based reasoning (CBR) [11, 18-20].”
MethodDecision Trees for Mining Data Streams Based on the McDiarmid's Bound
Rutkowski, L., Pietruczuk, L., Duda, P., & Jaworski, M. (2013) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2012.66 · Cited by 193
He, R., & McAuley, J. (2016) · arXiv · doi:10.48550/arxiv.1602.01585 · Cited by 186
“Such learning algorithms include decision trees [37], SVMs [18], instance-based learning [1], etc.”
MethodEvent-based classification of social media streams
Reuter, T., & Cimiano, P. (2012) · Proceedings of the 2nd ACM International Conference on Multimedia Retrieval · doi:10.1145/2324796.2324824 · Cited by 98
“This is a problem that has been referred to as ‘ ‘concept drift” [17, 35].”
BackgroundA new data mining based network Intrusion Detection model
Gudadhe, M., Prasad, P., & Kapil Wankhade, L. (2010) · 2010 International Conference on Computer and Communication Technology (ICCCT) · doi:10.1109/iccct.2010.5640375 · Cited by 59
Metadata
- Authors
- Wang, H., Fan, W., Yu, P. S., & Han, J.
- Venue
- Proceedings of the ninth ACM SIGKDD international conference on Knowledge discovery and data mining
- Publisher
- ACM
- Published
- 2003-08-24
- Pages
- 226-235
- OpenAlex
- W2009727399
- Cited by
- 1,256
- References
- 23
- FWCI
- 21.60
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.