The DOI matches its Crossref record; 57 of the 57 references it cites were found in a registry or an index.
Incremental Learning of Concept Drift in Nonstationary Environments
Abstract
We introduce an ensemble of classifiers-based approach for incremental learning of concept drift, characterized by nonstationary environments (NSEs), where the underlying data distributions change over time. The proposed algorithm, named Learn(++). NSE, learns from consecutive batches of data without making any assumptions on the nature or rate of drift; it can learn from such environments that experience constant or variable rate of drift, addition or deletion of concept classes, as well as cyclical drift. The algorithm learns incrementally, as other members of the Learn(++) family of algorithms, that is, without requiring access to previously seen data. Learn(++). NSE trains one new classifier for each batch of data it receives, and combines these classifiers using a dynamically weighted majority voting. The novelty of the approach is in determining the voting weights, based on each classifier's time-adjusted accuracy on current and past environments. This approach allows the algorithm to recognize, and act accordingly, to the changes in underlying data distributions, as well as to a possible reoccurrence of an earlier distribution. We evaluate the algorithm on several synthetic datasets designed to simulate a variety of nonstationary environments, as well as a real-world weather prediction dataset. Comparisons with several other approaches are also included. Results indicate that Learn(++). NSE can track the changing environments very closely, regardless of the type of concept drift. To allow future use, comparison and benchmarking by interested researchers, we also release our data used in this paper.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
(1978) · Medical Entomology and Zoology · Cited by 53,488
A decision-theoretic generalization of on-line learning and an application to boosting
Freund, Y., & Schapire, R. E. (1997) · Journal of Computer and System Sciences · doi:10.1006/jcss.1997.1504 · Cited by 20,466
REMEMBERING: A STUDY IN EXPERIMENTAL AND SOCIAL PSYCHOLOGY
Bartlett, F. C., & Burt, C. (1933) · British Journal of Educational Psychology · doi:10.1111/j.2044-8279.1933.tb02913.x · Cited by 7,475
The Strength of Weak Learnability
Schapire, R. E. (1990) · Machine Learning · doi:10.1007/bf00116037 · Cited by 4,600
Remembering: A Study in Experimental and Social Psychology
Bartlett, F. C., & Kintsch, W. (1995) · Cambridge University Press eBooks · doi:10.1017/cbo9780511759185 · Cited by 2,376
Learning in the Presence of Concept Drift and Hidden Contexts
Widmer, G., & Kubat, M. (1996) · Machine Learning · doi:10.1023/a:1018046501280 · Cited by 1,857
Mining time-changing data streams
Hulten, G., Spencer, L., & Domingos, P. (2001) · Proceedings of the seventh ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/502512.502529 · Cited by 1,748
Nonlinear neural networks: Principles, mechanisms, and architectures
Grossberg, S. (1988) · Neural Networks · doi:10.1016/0893-6080(88)90021-4 · Cited by 1,623
Dataset Shift in Machine Learning
(2008) · doi:10.7551/mitpress/9780262170055.001.0001 · Cited by 1,553
Gama, J., Medas, P., Castillo, G., & Rodrigues, P. (2004) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-28645-5_29 · Cited by 1,536
Piaget, J., Elkind, D., & Tenzer, A. (1967) · Medical Entomology and Zoology · Cited by 1,435
Mining concept-drifting data streams using ensemble classifiers
Wang, H., Fan, W., Yu, P. S., & Han, J. (2003) · Proceedings of the ninth ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/956750.956778 · Cited by 1,256
A streaming ensemble algorithm (SEA) for large-scale classification
Street, W. N., & Kim, Y. (2001) · Proceedings of the seventh ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/502512.502568 · Cited by 1,205
Scaffolding Complex Learning: The Mechanisms of Structuring and Problematizing Student Work
Reiser, B. J. (2004) · Journal of the Learning Sciences · doi:10.1207/s15327809jls1303_2 · Cited by 1,054
Learn++: an incremental learning algorithm for supervised neural networks
Polikar, R., Upda, L., Upda, S. S., & Honavar, V. (2001) · IEEE Transactions on Systems, Man and Cybernetics, Part C (Applications and Reviews) · doi:10.1109/5326.983933 · Cited by 888
Self-citationNovelty detection: a review—part 2:
Markou, M., & Singh, S. (2003) · Signal Processing · doi:10.1016/j.sigpro.2003.07.019 · Cited by 817
BifetAlbert, HolmesGeoff, KirkbyRichard, & PfahringerBernhard (2010) · Journal of Machine Learning Research · doi:10.5555/1756006.1859903 · Cited by 771
Dataset Shift in Machine Learning
Adams, N. (2010) · Journal of the Royal Statistical Society Series A: Statistics in Society · doi:10.1111/j.1467-985x.2009.00624_10.x · Cited by 694
Incremental learning from noisy data
Schlimmer, J. C., & Granger, R. H. (1986) · Machine Learning · doi:10.1007/bf00116895 · Cited by 653
Dynamic Weighted Majority: An Ensemble Method for Drifting Concepts
Kolter, J. Z., & Maloof, M. A. (2007) · doi:10.5555/1314498.1390333 · Cited by 648
Learning quickly when irrelevant attributes abound: A new linear-threshold algorithm
Littlestone, N. (1988) · Machine Learning · doi:10.1007/bf00116827 · Cited by 497
The Impact of Diversity on Online Ensemble Learning in the Presence of Concept Drift
Minku, L. L., White, A. P., & Yao, X. (2010) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2009.156 · Cited by 440
Learning drifting concepts: Example selection vs. example weighting
Klinkenberg, R. (2004) · Intelligent Data Analysis · doi:10.3233/ida-2004-8305 · Cited by 438
Dynamic weighted majority: a new ensemble method for tracking concept drift
Kolter, J. Z., & Maloof, M. A. (2004) · Third IEEE International Conference on Data Mining · doi:10.1109/icdm.2003.1250911 · Cited by 419
Mining concept-drifting data streams using ensemble classifiers
Wang, H., Fan, W., Yu, P. S., & Han, J. (2003) · ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/956755.956778 · Cited by 390
Classifier Ensembles for Changing Environments
Kuncheva, L. I. (2004) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-25966-4_1 · Cited by 335
A General Framework for Mining Concept-Drifting Data Streams with Skewed Distributions
Gao, J., Fan, W., Han, J., & Yu, P. S. (2007) · Proceedings of the 2007 SIAM International Conference on Data Mining · doi:10.1137/1.9781611972771.1 · Cited by 242
Muhlbaier, M. D., Topalis, A., & Polikar, R. (2009) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2008.2008326 · Cited by 207
Self-citationAdaptive Probabilistic Neural Networks for Pattern Classification in Time-Varying Environment
Rutkowski, L. (2004) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.828757 · Cited by 202
Dynamic integration of classifiers for handling concept drift
Tsymbal, A., Pechenizkiy, M., Cunningham, P., & Puuronen, S. (2008) · Information Fusion · doi:10.1016/j.inffus.2006.11.002 · Cited by 195
Blum, A. (1997) · Machine Learning · doi:10.1023/a:1007335615132 · Cited by 191
Oza, N. C. (2000) · Cited by 189
Just-in-Time Adaptive Classifiers—Part I: Detecting Nonstationary Changes
Alippi, C., & Roveri, M. (2008) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2008.2000082 · Cited by 154
Chen, S., & He, H. (2010) · Evolving Systems · doi:10.1007/s12530-010-9021-y · Cited by 144
On Appropriate Assumptions to Mine Data Streams: Analysis and Practice
Gao, J., Fan, W., & Han, J. (2007) · Proceedings · doi:10.1109/icdm.2007.96 · Cited by 141
Flavell, J. H. (1996) · Psychological Science · doi:10.1111/j.1467-9280.1996.tb00359.x · Cited by 135
Classifying Data Streams with Skewed Class Distributions and Concept Drifts
Gao, J., Ding, B., Fan, W., et al. (2008) · IEEE Internet Computing · doi:10.1109/mic.2008.119 · Cited by 129
Codebook-Based Bayesian Speech Enhancement for Nonstationary Environments
Srinivasan, S., Samuelsson, J., & Kleijn, W. B. (2007) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2006.881696 · Cited by 126
Classifier Ensembles for Detecting Concept Change in Streaming Data: Overview and Perspectives
Kuncheva, L. I. (2008) · Cited by 110
Boosting classifiers for drifting concepts
Klinkenberg, R., & Scholz, M. (2006) · Technische Universität Dortmund Eldorado (Technische Universität Dortmund) · doi:10.17877/de290r-14320 · Cited by 106
Entropy-based Concept Shift Detection
Vorburger, P., & Bernstein, A. (2006) · Proceedings · doi:10.1109/icdm.2006.66 · Cited by 96
Boosting classifiers for drifting concepts
Scholz, M., & Klinkenberg, R. (2007) · Intelligent Data Analysis · doi:10.3233/ida-2007-11102 · Cited by 89
Learn++.MF: A random subspace approach for the missing feature problem
Robi Polikar, A., DePasquale, J., Mohammed, H. S., Gavin Brown, B., & Kuncheva, L. I. (2010) · Pattern Recognition · doi:10.1016/j.patcog.2010.05.028 · Cited by 88
Adaptive learning and mining for data streams and frequent patterns
Bifet, A. (2009) · ACM SIGKDD Explorations Newsletter · doi:10.1145/1656274.1656287 · Cited by 86
Just-in-Time Adaptive Classifiers—Part II: Designing the Classifier
Alippi, C., & Roveri, M. (2008) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2008.2003998 · Cited by 82
Advances in Artificial Intelligence
Shi, D., Gao, J., So Yeung, D., & Chen, F. (2004) · Lecture notes in computer science · doi:10.1007/b97823 · Cited by 81
Dynamic Classifier Selection for Effective Mining from Noisy Data Streams
Xingquan Zhu, A., Wu, X., & Yang, Y. (2005) · Proceedings · doi:10.1109/icdm.2004.10091 · Cited by 67
Incremental learning in nonstationary environments with controlled forgetting
Elwell, R., & Polikar, R. (2009) · doi:10.1109/ijcnn.2009.5178779 · Cited by 61
Self-citationTsymbal, A., Pechenizkiy, M., Cunningham, P., & Puuronen, S. J. (2006) · Proceedings - IEEE Symposium on Computer-Based Medical Systems · doi:10.1109/cbms.2006.94 · Cited by 54
IMORL: Incremental Multiple-Object Recognition and Localization
He, H., & Chen, S. (2008) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2008.2001774 · Cited by 53
Adaptive Classifiers-Ensemble System for Tracking Concept Drift
Nishida, K., & Yamauchi, K. (2007) · International Conference on Machine Learning and Cybernetics · doi:10.1109/icmlc.2007.4370772 · Cited by 49
Learning concept drift in nonstationary environments using an ensemble of classifiers based approach
Karnick, M. T., Ahiskali, M., Muhlbaier, M. D., & Robi Polikar, A. (2008) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2008.4634290 · Cited by 42
: An adaptive classifier for data streams
Pavlidis, N. G., Tasoulis, D. K., Adams, N. M., & Hand, D. J. (2010) · Pattern Recognition · doi:10.1016/j.patcog.2010.07.026 · Cited by 38
Karnick, M. T., Muhlbaier, M. D., & Robi Polikar, A. (2008) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr.2008.4761062 · Cited by 28
Sequential non-stationary dynamic classification with sparse feedback
Lowne, D. R., Roberts, S., & Garnett, R. (2009) · Pattern Recognition · doi:10.1016/j.patcog.2009.09.004 · Cited by 20
Equilibration : theory, research, and application
Appel, M. H., & Goldberg, L. S. (1977) · Medical Entomology and Zoology · Cited by 7
The Rosenblatt Bayesian Algorithm Learning in a Nonstationary Environment
de Oliveira, E. A. (2007) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2006.889943 · Cited by 6
Cited by
A survey on concept drift adaptation
Gama, J., Žliobaitė, I., Bifet, A., Pechenizkiy, M., & Bouchachia, A. (2014) · ACM Computing Surveys · doi:10.1145/2523813 · Cited by 3,735
Learning under Concept Drift: A Review
Lu, J., Liu, A., Dong, F., Gu, F., Gama, J., & Zhang, G. (2018) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2018.2876857 · Cited by 1,603
Big data deep learning: Challenges and perspectives
Chen, X.-W., & Lin, X. (2014) · IEEE Access · doi:10.1109/access.2014.2325029 · Cited by 1,255
Ensemble learning for data stream analysis: A survey
Krawczyk, B., Minku, L. L., Gama, J., Stefanowski, J., & Woźniak, M. (2017) · Information Fusion · doi:10.1016/j.inffus.2017.02.004 · Cited by 1,050
Online learning: A comprehensive survey
Hoi, S. C. H., Sahoo, D., Lu, J., & Zhao, P. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.04.112 · Cited by 746
Metadata
- Authors
- Elwell, R., & Polikar, R.
- Venue
- IEEE Transactions on Neural Networks
- Published
- 2011-10
- Volume
- 22
- Issue
- 10
- Pages
- 1517-1531
- ISSN
- 1045-9227 (ISSN-L)
- 1941-0093
- OpenAlex
- W2160512933
- Cited by
- 931
- References
- 57
- FWCI
- 36.96
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.