The DOI matches its Crossref record; 42 of the 42 references it cites were found in a registry or an index.
Dealing with Noise Problem in Machine Learning Data-sets: A Systematic Review
Open accessAbstract
The occurrences of noisy data in data set can significantly impact prediction of any meaningful information. Many empirical studies have shown that noise in data set dramatically led to decreased classification accuracy and poor prediction results. Therefore, the problem of identifying and handling noise in prediction application has drawn considerable attention over past many years. In our study, we performed a systematic literature review of noise identification and handling studies published in various conferences and journals between January 1993 to July 2018. We have identified 79 primary studies are of noise identification and noise handling techniques. After investigating these studies, we found that among the noise identification schemes, the accuracy of identification of noisy instances by using ensemble-based techniques are better than other techniques. But regarding efficiency, usually single based techniques method is better; it is more suitable for noisy data sets. Among noise handling techniques, polishing techniques generally improve classification accuracy than filtering and robust techniques, but it introduced some errors in the data sets.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Class Noise vs. Attribute Noise: A Quantitative Study
Zhu, X., & Wu, X. (2004) · Artificial Intelligence Review · doi:10.1007/s10462-004-0751-8 · Cited by 893
Dealing with noise in defect prediction
Kim, S., Zhang, H., Wu, R., & Gong, L. (2011) · Proceedings of the 33rd International Conference on Software Engineering · doi:10.1145/1985793.1985859 · Cited by 330
Comparing Boosting and Bagging Techniques With Noisy and Imbalanced Data
Khoshgoftaar, T. M., Van Hulse, J., & Napolitano, A. (2011) · IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans · doi:10.1109/tsmca.2010.2084081 · Cited by 300
Pelletier, C., Valero, S., Inglada, J., Champion, N., Marais Sicre, C., & Dedieu, G. (2017) · Remote Sensing · doi:10.3390/rs9020173 · Cited by 214
Repeated labeling using multiple noisy labelers
Ipeirotis, P. G., Provost, F., Sheng, V. S., & Wang, J. (2014) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-013-0306-1 · Cited by 181
Ensemble Methods for Noise Elimination in Classification Problems
Verbaeten, S., & Van Assche, A. (2003) · Lecture Notes in Computer Science · doi:10.1007/3-540-44938-8_32 · Cited by 168
Effect of label noise in the complexity of classification problems
Garcia, L. P. F., de Carvalho, A. C. P. L. F., & Lorena, A. C. (2015) · Neurocomputing · doi:10.1016/j.neucom.2014.10.085 · Cited by 148
A noise-detection based AdaBoost algorithm for mislabeled data
Cao, J., Kwong, S., & Wang, R. (2012) · Pattern Recognition · doi:10.1016/j.patcog.2012.05.002 · Cited by 147
Identifying and Handling Mislabelled Instances
Muhlenbach, F., Lallich, S., & Zighed, A. D. (2003) · Journal of Intelligent Information Systems · doi:10.1023/a:1025832930864 · Cited by 142
Noise detection and elimination in data preprocessing: Experiments in medical domains
Gamberger, D., Nada Lavrač, A., & Džeroski, S. (2000) · Applied Artificial Intelligence · doi:10.1080/088395100117124 · Cited by 135
Noisy data elimination using mutual k-nearest neighbor for classification mining
Liu, H., & Zhang, S. (2011) · Journal of Systems and Software · doi:10.1016/j.jss.2011.12.019 · Cited by 120
Improving classification accuracy by identifying and removing instances that should be misclassified
Smith, M. R., & Martinez, T. R. (2011) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2011.6033571 · Cited by 119
Class Noise Mitigation Through Instance Weighting
Rebbapragada, U. D., & Brodley, C. E. (2007) · Lecture notes in computer science · doi:10.1007/978-3-540-74958-5_71 · Cited by 98
Sáez, J. A., Galar, M., Luengo, J., & Herrera, F. (2015) · Information Fusion · doi:10.1016/j.inffus.2015.04.002 · Cited by 94
Experiments on Ensembles with Missing and Noisy Data
Melville, P., Shah, N., Mihalkova, L. S., & Mooney, R. J. (2004) · Lecture notes in computer science · doi:10.1007/978-3-540-25966-4_29 · Cited by 92
Robust ensemble learning for mining noisy data streams
Zhang, P., Xingquan Zhu, A., Shi, Y., Guo, L., & Wu, X. (2010) · Decision Support Systems · doi:10.1016/j.dss.2010.11.004 · Cited by 89
Ensemble-based noise detection: noise ranking and visual performance evaluation
Sluban, B., Gamberger, D., & Lavrač, N. (2013) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-012-0299-1 · Cited by 88
Sáez, J. A., Luengo, J., & Herrera, F. (2015) · Neurocomputing · doi:10.1016/j.neucom.2014.11.086 · Cited by 88
Data Cleaning for Classification Using Misclassification Analysis
Jeatrakul, P., Wong, K. W., & Fung, C. C. (2010) · Journal of Advanced Computational Intelligence and Intelligent Informatics · doi:10.20965/jaciii.2010.p0297 · Cited by 85
Use of Classification Algorithms in Noise Detection and Elimination
Miranda, A., Garcia, L. P. F., de Carvalho, A. C. P. L. F., & Lorena, A. C. (2009) · Lecture notes in computer science · doi:10.1007/978-3-642-02319-4_50 · Cited by 82
The pairwise attribute noise detection algorithm
Hulse, J. V., Khoshgoftaar, T. M., & Huang, H. (2006) · Knowledge and Information Systems · doi:10.1007/s10115-006-0022-x · Cited by 77
Enhancing software quality estimation using ensemble-classifier based noise filtering
Khoshgoftaar, T. M., Zhong, S. S., & Joshi, V. H. (2005) · Intelligent Data Analysis · doi:10.3233/ida-2005-9102 · Cited by 76
Can PAC learning algorithms tolerate random attribute noise?
Goldman, S. A., & Sloan, R. H. (1995) · Algorithmica · doi:10.1007/bf01300374 · Cited by 68
Noise detection in the meta-learning level
Garcia, L. P. F., de Carvalho, A. C. P. L. F., & Lorena, A. C. (2015) · Neurocomputing · doi:10.1016/j.neucom.2014.12.100 · Cited by 67
Identifying and Correcting Mislabeled Training Instances
Sun, J., Zhao, F., Wang, C., & Chen, S. (2007) · International Conference on Future Generation Communication and Networking · doi:10.1109/fgcn.2007.146 · Cited by 66
Identifying learners robust to low quality data
Folleco, A., Khoshgoftaar, T. M., Hulse, J. V., & Bullard, L. (2008) · IEEE International Conference on Information Reuse and Integration (IRI) · doi:10.1109/iri.2008.4583028 · Cited by 64
Identifying mislabeled training data with the aid of unlabeled data
Guan, D., Yuan, W., Lee, Y.-K., & Lee, S. (2010) · Applied Intelligence · doi:10.1007/s10489-010-0225-4 · Cited by 55
HYDRA: A Noise-tolerant Relational Concept Learning Algorithm.
Ali, K., & Pazzani, M. J. (1993) · eScholarship (California Digital Library) · Cited by 54
A Comparison of Noise Handling Techniques
Teng, C.-M. (2001) · Cited by 53
Noise reduction for instance-based learning with a local maximal margin approach
Segata, N., Blanzieri, E., Delany, S. J., & Cunningham, P. (2009) · Journal of Intelligent Information Systems · doi:10.1007/s10844-009-0101-z · Cited by 52
Xingquan Zhu, A., & Wu, X. (2006) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2006.155 · Cited by 51
Xingquan Zhu, A., Wu, X., & Chen, Q. (2006) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-005-0012-8 · Cited by 45
Pre-processing for noise detection in gene expression classification data
Libralon, G. L., de Carvalho, A. C. P. L. F., & Lorena, A. C. (2009) · Journal of the Brazilian Computer Society · doi:10.1007/bf03192573 · Cited by 41
Empirical Case Studies in Attribute Noise Detection
Khoshgoftaar, T. M., & Hulse, J. V. (2009) · IEEE Transactions on Systems Man and Cybernetics Part C (Applications and Reviews) · doi:10.1109/tsmcc.2009.2013815 · Cited by 38
A Study on Class Noise Detection and Elimination
Garcia, L. P. F., Lorena, A. C., & de Carvalho, A. C. P. L. F. (2012) · Brazilian Symposium on Neural Networks (SBRN) · doi:10.1109/sbrn.2012.49 · Cited by 37
Teng, C. M. (2000) · Lecture notes in computer science · doi:10.1007/3-540-44533-1_22 · Cited by 35
An algorithm for correcting mislabeled data
Zeng, X., & Martinez, T. R. (2001) · Intelligent Data Analysis · doi:10.3233/ida-2001-5605 · Cited by 34
Classification algorithm sensitivity to training data with non representative attribute noise
Mannino, M. V., Yang, Y., & Ryu, Y. U. (2008) · Decision Support Systems · doi:10.1016/j.dss.2008.11.021 · Cited by 33
Evaluation of noise reduction techniques in the splice junction recognition problem
Lorena, A. C., & de Carvalho, A. C. P. L. F. (2004) · Genetics and Molecular Biology · doi:10.1590/s1415-47572004000400031 · Cited by 31
Classification with class noises through probabilistic sampling
Yuan, W., Guan, D., Ma, T., & Khattak, A. M. (2017) · Information Fusion · doi:10.1016/j.inffus.2017.08.007 · Cited by 21
Identifying noisy features with the Pairwise Attribute Noise Detection Algorithm
Khoshgoftaar, T. M., & Hulse, J. V. (2005) · Intelligent Data Analysis · doi:10.3233/ida-2005-9606 · Cited by 10
Ensembles of Pre-processing Techniques for Noise Detection in Gene Expression Data
Libralon, G. L., de Carvalho, A. C. P. L. F., & Lorena, A. C. (2009) · Lecture notes in computer science · doi:10.1007/978-3-642-02490-0_60 · Cited by 9
Cited by
A review: Data pre-processing and data augmentation techniques
Maharana, K., Mondal, S., & Nemade, B. (2022) · Global Transitions Proceedings · doi:10.1016/j.gltp.2022.04.020 · Cited by 1,337
Sharma, A., Mukhopadhyay, T., Rangappa, S. M., Siengchin, S., & Kushvaha, V. (2022) · Archives of Computational Methods in Engineering · doi:10.1007/s11831-021-09700-9 · Cited by 285
Young, R. A., Martin, C. M., Sturmberg, J. P., Hall, S., Bazemore, A., Kakadiaris, I. A., & Lin, S. (2024) · The Journal of the American Board of Family Medicine · doi:10.3122/jabfm.2023.230219r1 · Cited by 17
Methods and Annotated Data Sets Used to Predict the Gender and Age of Twitter Users: Scoping Review
O'Connor, K., Golder, S., Weissenbacher, D., Klein, A. Z., Magge, A., & Gonzalez-Hernandez, G. (2024) · Journal of Medical Internet Research · doi:10.2196/47923 · Cited by 10
Metadata
- Authors
- Gupta, S., & Gupta, A.
- Venue
- Procedia Computer Science
- Publisher
- Elsevier BV
- Published
- 2019
- Volume
- 161
- Pages
- 466-474
- ISSN
- 1877-0509 (ISSN-L)
- OpenAlex
- W2997655715
- Cited by
- 325
- References
- 42
- FWCI
- 7.71
- Type
- conference paper
- Language
- en
- Open access
- diamond
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.