The DOI matches its Crossref record; 65 of the 65 references it cites were found in a registry or an index.
Diversity creation methods: a survey and categorisation
Abstract
Ensemble approaches to classication and regression have attracted a great deal of interest in recent years. These methods can be shown both theoretically and empirically to outperform single predictors on a wide range of tasks. One of the elements required for accurate prediction when using an ensemble is recognised to be error \diversity". However, the exact meaning of this concept is not clear from the literature, particularly for classication tasks. In this paper we rst review the varied attempts to provide a formal explanation of error diversity, including several heuristic and qualitative explanations in the literature. For completeness of discussion we include not only the classication literature but also some excerpts of the rather more mature regression literature, which we believe can still provide some insights. We proceed to survey the various techniques used for creating diverse ensembles, and categorise them, forming a preliminary taxonomy of diversity creation methods. As part of this taxonomy we introduce the idea of implicit and explicit diversity creation methods, and three dimensions along which these may be applied. Finally we propose some new directions that may prove fruitful in understanding classication error diversity.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Neural Networks: A Comprehensive Foundation
Haykin, S. (1998) · Cited by 30,070
Breiman, L. (1996) · Machine Learning · doi:10.1007/bf00058655 · Cited by 16,325
Genetic Programming: On the Programming of Computers by Means of Natural Selection
Koza, J. R. (1992) · Medical Entomology and Zoology · Cited by 13,082
Proceedings of the 24th international conference on Machine learning
(2007) · doi:10.1145/1273496 · Cited by 11,630
Solutions of Ill-Posed Problems.
Bell, J. B., Tikhonov, A. N., & Arsenin, V. Y. (1978) · Mathematics of Computation · doi:10.2307/2006360 · Cited by 8,101
Neural networks: A comprehensive foundation
Tadeusiewicz, R. (1995) · Control Engineering Practice · doi:10.1016/0967-0661(95)90080-2 · Cited by 7,218
Hansen, L. K., & Salamon, P. (1990) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.58871 · Cited by 4,260
Neural Networks and the Bias/Variance Dilemma
Geman, S., Bienenstock, E., & Doursat, R. (1992) · Neural Computation · doi:10.1162/neco.1992.4.1.1 · Cited by 3,578 · cited 2×
“However, this can in fact be shown more simply by the same manipulations as used in the bias–variance decomposition [11], reflecting a strong relationship between the two decompositions.”
BackgroundMethodBates, J. M., & Granger, C. W. J. (1969) · Journal of the Operational Research Society · doi:10.1057/jors.1969.103 · Cited by 3,201
Popular Ensemble Methods: An Empirical Study
Opitz, D., & Maclin, R. (1999) · Journal of Artificial Intelligence Research · doi:10.1613/jair.614 · Cited by 2,953
Measures of Diversity in Classifier Ensembles and Their Relationship with the Ensemble Accuracy
Kuncheva, L. I., & Whitaker, C. J. (2003) · Machine Learning · doi:10.1023/a:1022859003006 · Cited by 2,335
Combining forecasts: A review and annotated bibliography
Clemen, R. T. (1989) · International Journal of Forecasting · doi:10.1016/0169-2070(89)90012-5 · Cited by 2,164
Neural Network Ensembles, Cross Validation, and Active Learning
Krogh, A., & Vedelsby, J. (1994) · Cited by 1,838
Combining forecasts: A review and annotated bibliography
(1990) · Long Range Planning · doi:10.1016/0024-6301(90)90316-v · Cited by 1,465
On Bias, Variance, 0/1—Loss, and the Curse-of-Dimensionality
Friedman, J. H. (1997) · Data Mining and Knowledge Discovery · doi:10.1023/a:1009778005914 · Cited by 1,040
Combination of multiple classifiers using local accuracy estimates
Woods, K., Kegelmeyer, W. P., & Bowyer, K. (1997) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.588027 · Cited by 952
Ensemble learning via negative correlation
Liu, Y., & Yao, X. (1999) · Neural Networks · doi:10.1016/s0893-6080(99)00073-8 · Cited by 712
Self-citationBias plus variance decomposition for zero-one loss functions
Kohavi, R., & Wolpert, D. H. (1996) · Cited by 652
Error Correlation and Error Reduction in Ensemble Classifiers
TUMER, K., & GHOSH, J. (1996) · Connection Science · doi:10.1080/095400996116839 · Cited by 597
Margineantu, D. D., & Dietterich, T. G. (1997) · Cited by 553
Optimal Linear Combinations of Neural Networks
Hashem, S. (1997) · Neural Networks · doi:10.1016/s0893-6080(96)00098-6 · Cited by 499
Experiment with a new boosting algorithm
Freund, Y. (1996) · Medical Entomology and Zoology · Cited by 497
Combining Artificial Neural Nets: Ensemble and Modular Multi-Net Systems
SHARKEY, A. J. C. (1999) · Cited by 482
Error-Correcting Output Coding Corrects Bias and Variance
Kong, E. B., & Dietterich, T. G. (1995) · Machine Learning Proceedings 1995 · doi:10.1016/b978-1-55860-377-6.50046-3 · Cited by 365
Limits on the majority vote accuracy in classifier fusion
Kuncheva, L. I., Whitaker, C. J., Shipp, C. A., & Duin, R. P. W. (2003) · Pattern Analysis & Applications · doi:10.1007/s10044-002-0173-7 · Cited by 356
Learning with Ensembles: How over-fitting can be useful
Peter Sollich, A., & Krogh, A. (1995) · Cited by 353
Analysis of decision boundaries in linearly combined neural classifiers
Tumer, K., & Ghosh, J. (1996) · Pattern Recognition · doi:10.1016/0031-3203(95)00085-2 · Cited by 335
Invited review combining forecasts—twenty years later
Granger, C. W. J. (1989) · Journal of Forecasting · doi:10.1002/for.3980080303 · Cited by 322
Feature selection for ensembles
Opitz, D. W. (1999) · Cited by 310
A constructive algorithm for training cooperative neural network ensembles
Islam, M. M., Yao, X., & Murase, K. (2003) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2003.813832 · Cited by 294
Self-citationGenerating Accurate and Diverse Members of a Neural-Network Ensemble
Opitz, D. W., & SHAVLIK, J. W. (1995) · Cited by 279
Randomizing Outputs to Increase Prediction Accuracy
Breiman, L. (2000) · Machine Learning · doi:10.1023/a:1007682208299 · Cited by 276
Relationships between combination methods and measures of diversity in combining classifiers
Shipp, C. A., & Kuncheva, L. I. (2002) · Information Fusion · doi:10.1016/s1566-2535(02)00051-9 · Cited by 270
Constructing diverse classifier ensembles using artificial training examples
Melville, P., & Mooney, R. J. (2003) · Cited by 263
Making use of population information in evolutionary artificial neural networks
Yao, X., & Liu, Y. (1998) · IEEE Transactions on Systems, Man and Cybernetics, Part B (Cybernetics) · doi:10.1109/3477.678637 · Cited by 257
Self-citationEnsemble Learning Using Decorrelated Neural Networks
ROSEN, B. E. (1996) · Connection Science · doi:10.1080/095400996116820 · Cited by 248
Experiments with Classifier Combining Rules
Duin, R. P. W., & Tax, D. M. J. (2000) · Lecture Notes in Computer Science · doi:10.1007/3-540-45014-9_2 · Cited by 235
Generalization error of ensemble estimators
Ueda, N., & Nakano, R. (2002) · Proceedings of International Conference on Neural Networks (ICNN'96) · doi:10.1109/icnn.1996.548872 · Cited by 230
Diversity versus Quality in Classification Ensembles Based on Feature Selection
Cunningham, P., & Carney, J. G. (2000) · Lecture notes in computer science · doi:10.1007/3-540-45164-1_12 · Cited by 219
Engineering Multiversion Neural-Net Systems
Partridge, D., & Yates, W. B. (1996) · Neural Computation · doi:10.1162/neco.1996.8.4.869 · Cited by 218
Error-Correcting Output Codes: A General Method for Improving Multiclass Inductive Learning Programs
Dietterich, T. G. (2018) · The Mathematics of Generalization · doi:10.1201/9780429492525-14 · Cited by 207
Random Forests--Random Features
Breiman, L. (1999) · Cited by 204
Is independence good for combining classifiers?
Kuncheva, L. I., Whitaker, C. J., Shipp, C. A., & Duin, R. P. W. (2002) · Proceedings 15th International Conference on Pattern Recognition. ICPR-2000 · doi:10.1109/icpr.2000.906041 · Cited by 194
Bootstrapping with Noise: An Effective Regularization Technique
RAVIV, Y., & INTRATOR, N. (1996) · Connection Science · doi:10.1080/095400996116811 · Cited by 189
Zenobi, G., & Cunningham, P. (2001) · Lecture Notes in Computer Science · doi:10.1007/3-540-44795-4_49 · Cited by 183
A Unifeid Bias-Variance Decomposition and its Applications
Domingos, P. (2000) · International Conference on Machine Learning · Cited by 167
Variance and Bias for General Loss Functions
James, G. M. (2003) · Machine Learning · doi:10.1023/a:1022899518027 · Cited by 160
Perrone, M. (1993) · Cited by 159
SHARKEY, A. J. C., & SHARKEY, N. E. (1997) · The Knowledge Engineering Review · doi:10.1017/s0269888997003123 · Cited by 159
Diversity in Neural Network Ensembles
Gavin Brown, B. (2004) · Cited by 153
Ten measures of diversity in classifier ensembles: limits for two classifiers
Kuncheva, L. I. (2001) · DERA/IEE Workshop on Intelligent Sensor Processing · doi:10.1049/ic:20010105 · Cited by 152
That Elusive Diversity in Classifier Ensembles
Kuncheva, L. I. (2003) · Lecture notes in computer science · doi:10.1007/978-3-540-44871-6_130 · Cited by 134
Speciation as automatic categorical modularization
Darwen, P. J., & Yao, X. (1997) · IEEE Transactions on Evolutionary Computation · doi:10.1109/4235.687878 · Cited by 120
Self-citationSHARKEY, A. J. C. (1999) · Perspectives in neural computing · doi:10.1007/978-1-4471-0793-4_1 · Cited by 117
Maclin, R., & SHAVLIK, J. W. (1995) · Cited by 109
The “Test and Select” Approach to Ensemble Combination
SHARKEY, A. J. C., SHARKEY, N. E., Gerecke, U., & Chandroth, G. O. (2000) · Lecture notes in computer science · doi:10.1007/3-540-45014-9_3 · Cited by 93
Improving Committee Diagnosis with Resampling Techniques
Parmanto, B., Munro, P., & Doyle, H. R. (1995) · Cited by 92
Input Decimation Ensembles: Decorrelation through Dimensionality Reduction
Oza, N. C., & Tumer, K. (2001) · Lecture notes in computer science · doi:10.1007/3-540-48219-9_24 · Cited by 91
Combining Decision Trees and Neural Networks for Drug Discovery
Langdon, W. B., Barrett, S. J., & Buxton, B. F. (2002) · Lecture notes in computer science · doi:10.1007/3-540-45984-7_6 · Cited by 84
Boosting with Averaged Weight Vectors
Oza, N. C. (2003) · Lecture notes in computer science · doi:10.1007/3-540-44938-8_2 · Cited by 61
Data Complexity Analysis for Classifier Combination
Ho, T. K. (2001) · Lecture notes in computer science · doi:10.1007/3-540-48219-9_6 · Cited by 58
Diversity between Neural Networks and Decision Trees for Building Multiple Classifier Systems
Wang, W., Jones, P., & Partridge, D. (2000) · Lecture notes in computer science · doi:10.1007/3-540-45014-9_23 · Cited by 48
Skurichina, M., Kuncheva, L. I., & Duin, R. P. W. (2002) · Lecture notes in computer science · doi:10.1007/3-540-45428-4_6 · Cited by 47
Use of methodological diversity to improve neural network generalisation
Yates, W. B., & Partridge, D. (1996) · Neural Computing and Applications · doi:10.1007/bf01413747 · Cited by 28
(1996)
Cited by
Rokach, L. (2010) · Artificial Intelligence Review · doi:10.1007/s10462-009-9124-7 · Cited by 2,657
Ensemble learning for data stream analysis: A survey
Krawczyk, B., Minku, L. L., Gama, J., Stefanowski, J., & Woźniak, M. (2017) · Information Fusion · doi:10.1016/j.inffus.2017.02.004 · Cited by 1,050
A survey of multiple classifier systems as hybrid systems
Woźniak, M., Graña, M., & Corchado, E. (2014) · Information Fusion · doi:10.1016/j.inffus.2013.04.006 · Cited by 912
Multiobjective Deep Belief Networks Ensemble for Remaining Useful Life Estimation in Prognostics
Zhang, C., Lim, P., Qin, A. K., & Tan, K. C. (2017) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2016.2582798 · Cited by 870
Ensemble approaches for regression: A survey
Mendes-Moreira, J., Soares, C., Jorge, A. M., & Sousa, J. F. D. (2012) · ACM Computing Surveys · doi:10.1145/2379776.2379786 · Cited by 667
Metadata
- Authors
- Brown, G., Wyatt, J., Harris, R., & Yao, X.
- Venue
- Information Fusion
- Publisher
- Elsevier BV
- Published
- 2005-03
- Volume
- 6
- Issue
- 1
- Pages
- 5-20
- ISSN
- 1566-2535 (ISSN-L)
- OpenAlex
- W2167055186
- Cited by
- 985
- References
- 65
- FWCI
- 31.52
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.