Data Evaluation and Enhancement for Quality Improvement of Machine Learning
The DOI matches its Crossref record; 57 of the 58 references it cites were found in a registry or an index.
Abstract
Poor data quality has a direct impact on the performance of the machine learning system that is built on the data. As a demonstrated effective approach for data quality improvement, transfer learning has been widely used to improve machine learning quality. However, the “quality improvement” brought by transfer learning was rarely rigorously validated, and some of the quality improvement results were misleading. This article first exposed the hidden quality problem in the datasets used to build a machine learning system for normalizing medical concepts in social media text. The system was claimed to have achieved the best performance compared to existing work on a machine learning task. However, the results of our experiments showed that the “best performance” was due to the poor quality of the datasets and the defective validation process. To address the data quality issue and build a high-performance medical concept normalization system, we developed a transfer-learning-based strategy for data quality enhancement and system performance improvement. The results of the experiments showed a strong correlation between the quality of the datasets and the performance of the machine learning system. The results also demonstrated that a rigorous evaluation of data quality is necessary for guiding the quality improvement of machine learning. Therefore, we propose a data quality evaluation framework that includes the quality criteria and their corresponding evaluation approaches. The data validation process, the performance improvement strategy, and the data quality evaluation framework discussed in this article can be used for machine learning researchers and practitioners to build high-performance machine learning systems.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Chen et al. [19] suggest that the thresholds can be based on the expected distribution of a targeted population for relevant features (e.g. the usage frequency of a phrase, or the number of individuals with a particular skin tone).”
From A Survey of Data Quality Requirements That Matter in ML Development Pipelines (2023) · cites this paper as [19]
In the citing paper
“For example, duplicate entries in a dataset can produce an overlap between the datasets used to train and evaluate a model, which can cause the performance metrics to be exaggerated [19].”
From A Survey of Data Quality Requirements That Matter in ML Development Pipelines (2023) · cites this paper as [19]
In the citing paper
“In cases like this, reusing and tuning a trained model helps to improve model performance only if the tuning is done using a dataset that contains task-specific data entries [19].”
From A Survey of Data Quality Requirements That Matter in ML Development Pipelines (2023) · cites this paper as [19]
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380
Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071
Distributed Representations of Words and Phrases and their Compositionality
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · cited 1×
“Therefore,thestate-of-the-artlanguagemodels,suchasBERT orGPT-3[41],weretrainedwithbillionsofwords.”
Kaugeranna, R., Kaugeranna, E., & Aion, (. S. 4. 6. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.giscience.2023.43 · Cited by 13,690
Learning Deep Architectures for AI
Bengio, Y. (2009) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000006 · Cited by 9,239
A Comprehensive Survey on Transfer Learning
Zhuang, F., Qi, Z., Duan, K., Xi, D., Zhu, Y., Zhu, H., Xiong, H., & He, Q. (2021) · Proceedings of the IEEE · doi:10.1109/jproc.2020.3004555 · Cited by 6,678 · cited 1×
“In order to show the performance of different transfer learning models, Zhuang et al. conductedanexperimentalstudyon20representativetransfer learning models, and its results demonstrated that appropriate transfer learning models should be carefully selected for different applications [62].”
BackgroundShin, H.-C., Roth, H. R., Gao, M., Lu, L., Xu, Z., Nogues, I., Yao, J., Mollura, D., & Summers, R. M. (2016) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2016.2528162 · Cited by 5,759
Learning Deep Architectures for AI
Bengio, Y. (2009) · now publishers, Inc. eBooks · doi:10.1561/9781601982957 · Cited by 4,941
Research works printed but not linked: 1
- belousov, 0, Using an ensemble of generalised linear and deep learning models in the SMM4H 2017 medical concept normalisation task, Proc 2nd Workshop Social Media Mining Health Appl, 54
Cited by
Items 101–166 of 166
Enhancing Object Detection in Aerial Images Using Transformer-Based Super-Resolution
Haykir, A. A., & Öksüz, İ. (2024) · International Conference on Computer Science and Engineering (UBMK) · doi:10.1109/ubmk63289.2024.10773484 · Cited by 3
Zhang, Y., Gao, S. S., Fan, X., Zhou, S., & Qu, J. (2024) · Bulletin of the Seismological Society of America · doi:10.1785/0120240179 · Cited by 2
Sun, X., Chen, L., Wang, Z., Wang, X., Wang, H., & Liu, L. (2024) · International Conference on Power System Technology (POWERCON) · doi:10.1109/powercon60995.2024.10870556 · Cited by 2
Improving prediction of water main failures using statistical and machine learning algorithms
Vaags, E. (2021) · cIRcle (University of British Columbia) · doi:10.14288/1.0402556 · Cited by 3
Computer Vision Based Phenotyping and Analysis in Presence of Uncertain Identification
Shumaly, M., Park, Y., Agha, S., Pandey, S., & Steibel, J. P. (2025) · Computers and Electronics in Agriculture · doi:10.1016/j.compag.2025.110560 · Cited by 2
Exploring the Influence of Data Characteristics on Machine Learning Outcomes
Sancricca, C., Castiglione, P., & Cappiello, C. (2025) · Lecture notes in business information processing · doi:10.1007/978-3-031-95397-2_14 · Cited by 4
Liang, X., Gu, H., Xu, Y., Wang, L., Guo, Y., & Liu, L. (2025) · Journal of Hydrology Regional Studies · doi:10.1016/j.ejrh.2025.102557 · Cited by 1
Baldini, G., & Kandeepan, S. (2025) · International Wireless Communications and Mobile Computing Conference (IWCMC) · doi:10.1109/iwcmc65282.2025.11059652 · Cited by 1
Jung, S., Moon, J., Suh, S., & Pyo, J. (2025) · Ecological Informatics · doi:10.1016/j.ecoinf.2025.103434 · Cited by 1
DCC: A Model-Free Frame to Evaluate Data Set Quality
Xie, C., & Li, Y. (2025) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.5c02429 · Cited by 1
Feng, A., Xu, L., Wang, Y., & Wu, Z. (2026) · Analytica Chimica Acta · doi:10.1016/j.aca.2026.345117 · Cited by 1
A Systematic Framework for Data Quality Assessment in Industrial Process Monitoring
Cao, L., Su, J., & Wang, Y. (2026) · IEEE Journal of Emerging and Selected Topics in Industrial Electronics · doi:10.1109/jestie.2026.3667011 · Cited by 2
PrimerAST: A predictive machine learning tool for primer design and quality assessment
Al-Mahrami, N., Yazidi, S. A., Alrashdi, H., Al-Jahwari, Z., Al-Mahrizi, M., Yousif, A. A., Al-Aamri, H. M., Al-Rahbi, B., & Jahdhami, N. A. (2026) · Scientific Reports · doi:10.1038/s41598-026-38238-8 · Cited by 2
Fan, M., Xiao, K., Sun, L., Li, N., Xu, Y., & Li, W. (2026) · Natural Resources Research · doi:10.1007/s11053-026-10681-4 · Cited by 1
A Novel Technique for Data Quality Improvement in Human Health PHM
Taco, J., Foreman, B. P., Kundu, P., & Lee, J. (2022) · Annual Conference of the PHM Society · doi:10.36001/phmconf.2022.v14i1.3177 · Cited by 1
Alexandrov, D., Zakharova, A., & Butakov, N. A. (2023) · International Journal of Semantic Computing · doi:10.1142/s1793351x24410046 · Cited by 1
Predicting spectroscopic properties of laser glasses: Theory and practice
Dong, S., Yao, J. I., Ming, H., Wang, W., & ZHANG, Q. (2024) · Scientia Sinica Technologica · doi:10.1360/sst-2024-0167 · Cited by 1
Nasef, D., Nasef, D., Sawiris, V., Girgis, P., & Toma, M. (2025) · BioMedInformatics · doi:10.3390/biomedinformatics5010003 · Cited by 1
Kamal, A. A., Mansor, M. A., & Kasihmuddin, M. S. M. (2025) · Mathematical Modeling and Computing · doi:10.23939/mmc2025.01.132 · Cited by 1
Collaborative Learning in Shared Production Environment Using Federated Image Classification
Hegiste, V., Legler, T., & Ruskowski, M. (2025) · Lecture notes in mechanical engineering · doi:10.1007/978-3-031-86489-6_11 · Cited by 1
Qu, H., Lian, W., Pang, S., Wu, Z., Ge, Y., & Wang, J. (2023) · Neural Computing and Applications · doi:10.1007/s00521-023-08316-3 · Cited by 1
Dataset Quality Assessment with Permutation Testing Showcased on Network Traffic DatasetsPreprint
Wasielewska, K., Soukup, D., Čejka, T., & Camacho, J. (2022) · doi:10.36227/techrxiv.20145539 · Cited by 2
Yamaguchi, T., Sasano, K., & Katsura, K. (2023) · Research Square · doi:10.21203/rs.3.rs-2615932/v1 · Cited by 1
Alexandrov, D., Zakharova, A., & Butakov, N. A. (2023) · IEEE International Conference on Semantic Computing (ICSC) · doi:10.1109/icsc56153.2023.00012
Training Node Screening in Decentralized Trusted Federated Learning
Wang, H., Jia-hua, Y., Xuan, S., & Li, X. (2023) · Lecture notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering · doi:10.1007/978-3-031-28990-3_17
AI-Supported Assessment of Load Safety
Schöning, J., & Kruse, N. (2023) · VDI Verlag eBooks · doi:10.51202/9783181024171-67
Sekine, M., Shimbara, D., Myojin, T., & Imatani, E. (2023) · IEEE International Conference on Artificial Intelligence Testing (AITest) · doi:10.1109/aitest58265.2023.00019
Yarozkaja, L., & Aleynikova, D. V. (2024) · doi:10.29039/978-5-94472-207-2-07-2024 · Cited by 1
Yan, A., Hu, K., & Wang, D. (2024) · International Journal of Machine Learning and Cybernetics · doi:10.1007/s13042-024-02320-2
Zhang, X., & Wang, B. (2024) · Applied Mathematics and Nonlinear Sciences · doi:10.2478/amns-2024-2741 · Cited by 1
Based on RuNewsCorp: Improving Accuracy in Long Text Classification
Sun, L., Aksyonov, K. A., & Wu, S. (2024) · International Russian Automation Conference (RusAutoCon) · doi:10.1109/rusautocon61949.2024.10694063 · Cited by 1
PREDICTION OF THE FRICTION COEFFICIENT BASED ON THE HYSTERESIS VALUE OF SHOE SOLE RUBBER
Nikolić, M. P., Banić, M., Pavlović, M., Pavlović, V., Miltenović, A., & Simonović, M. (2024) · Facta Universitatis Series Automatic Control and Robotics · doi:10.22190/fuacr241129009n
Predictive Modeling of Nanoscale Junctionless FinFET Using XGBoost
Ghoshhajra, R., Biswas, K., Sultana, M., & Sarkar, A. (2024) · IEEE International Conference of Electron Devices Society Kolkata Chapter (EDKCON) · doi:10.1109/edkcon62339.2024.10870617
Zhao, W., Qiu, X., Zhang, B., Li, B., Chen, J., & Wang, X. (2025) · Research Square · doi:10.21203/rs.3.rs-6319544/v1
Self-citationWang, Z., Yang, X. K., Li, T., Shu, L., Li, K., & Jing, X. (2025) · Electronics · doi:10.3390/electronics14153113 · Cited by 1
Luo, J., Du, Y., Wang, Y., Jiang, C., Yao, C., Zhang, X., Wang, L., Cun, D., & Ni, Q. (2025) · Journal of Applied Animal Welfare Science · doi:10.1080/10888705.2025.2542844
Battery Fault Detection Using Machine Learning: A Comprehensive Review
Saneep, K., Sundareswaran, K., Nayak, P. S. R., Simon, S. P., Josephine, R. A., & Puthusserry, G. V. (2025) · Transactions of Indian National Academy of Engineering · doi:10.1007/s41403-025-00549-z · Cited by 1
A Data-Driven Approach to Radio Frequency Signal Level Forecasting Using Machine Learning Algorithms
Giridhar, M., Priya, V. H. D., Devi, S., Saratha, B., Jaidharni, R., & Lakshmi, S. J. (2025) · International Conference on Inventive Systems and Control (ICISC) · doi:10.1109/icisc65841.2025.11187695
Stacked Neural Computing Model - Based Data Quality Analysis and Classification
Pratheesh, R., & Divya, V. (2025) · International Conference on Inventive Systems and Control (ICISC) · doi:10.1109/icisc65841.2025.11187811
ML model for predicting the intensity and frequency of urban building fires
Abbasi, A., Behnam, B., & Asadollahzadeh, D. (2025) · Sustainable and Resilient Infrastructure · doi:10.1080/23789689.2025.2574200
A Bioinspired Finger for Super-Resolution Dynamic Tactile Sensing
Ma, Y., Liang, L., Zhong, J., Zhu, Z., Zhang, T., & Wang, Z. (2025) · IEEE-RAS International Conference on Humanoid Robots (Humanoids) · doi:10.1109/humanoids65713.2025.11203146
Artificial Intelligence-Assisted General Ledger Anomaly Detection for Sustainable Auditing
Putra, M. D. N. A., & Chowanda, A. (2025) · International Conference on Information Technology Research and Innovation (ICITRI) · doi:10.1109/icitri67507.2025.11233117 · Cited by 2
Gearbox Failure Detection via Pre-Trained and Fine-Tuned Transformer Encoder
Puruncajas, B., España, S., Sanchez, J., Castellani, F., Tutivén, C., & Vidal, Y. (2025) · International Conference on Control and Fault-Tolerant Systems (SysTol) · doi:10.1109/systol66549.2025.11267351
Wasielewska, K., Soukup, D., Čejka, T., Carthy, J., & Camacho, J. (2025) · Applied Intelligence · doi:10.1007/s10489-025-06925-0
Data Detox: A Smart and Visual System for Anomaly Detection and Interactive Cleaning
Swathi, N., Shaik, F., Keerthi, H., Reddy, K. D., Sohail, A. M., & Khadeerulla, A. (2025) · International Conference on Emerging Applications of Material Science and Technology (ICEAMST) · doi:10.1109/iceamst67459.2025.11336048
Lu, F. (2025) · International Conference on Cyber Security and Information Engineering (ICCSIE) · doi:10.1109/iccsie65832.2025.00091
Ma, N., Qu, B., & Wang, W. (2026) · The European Journal on Artificial Intelligence · doi:10.1177/30504554261441546
Modeling and Measuring Redundancy in Multisource Multimodal Data for Autonomous Driving
Zhou, Y., Sattari, M., Chen, H., & Sha, K. (2026) · IEEE International Conference on Mobility, Operations, Services and Technologies (MOST) · doi:10.1109/most69733.2026.00011
Self-citationSun, H., Li, W., Wang, L., Wang, D., Liu, Y., Huang, W., & Wang, R. (2026) · IEEE Sensors Journal · doi:10.1109/jsen.2026.3704593
Improving the Quality of Enterprise Data Management with Tree-Based Models
Büyükbıçakcı, E. (2026) · Celal Bayar Üniversitesi Fen Bilimleri Dergisi · doi:10.18466/cbayarfbe.1677875
High Quality and Poor Quality in Machine Learning for Building Energy Prediction: A Review
Luo, C., Zhao, K., & Hong, S. H. (2026) · Sustainability · doi:10.3390/su18147275
Zhou, Z., He, Y. U. E. J. I., Guo, Y., Ren, T., & Yu, M. (2026) · Communications in computer and information science · doi:10.1007/978-981-92-3734-0_21
A systematic review of standards on bias and data quality from horizontal and vertical perspectives
Schmitz, A., & Poretschkin, M. (2026) · Journal of Data and Information Quality · doi:10.1145/3832313 · Cited by 1
Zhou, Y., Li, R., Cleveland, A. D., Chen, J., Sha, K., & Chen, H. (2026) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2026.105093 · Cited by 1
Self-citationLong, Y., Fang, H., Hou, B., Zhu, G., & Hursthouse, A. S. (2026) · Processes · doi:10.3390/pr14172847
A Comparative Benchmark of Biomedical Language Models for Concept Normalization from Real-World TextPreprint
Verma, A., Abhijay, Vangani, M., Prakash, S., & Chaudhary, K. (2026) · bioRxiv · doi:10.64898/2026.09.19.752843
Data-Centric AI: The Role of Data Quality Dimensions and Dataset Characteristics
Sancricca, C., & Cappiello, C. (2026) · SpringerBriefs in applied sciences and technology · doi:10.1007/978-3-032-29481-4_2
Generative AI for simulating brain development and aging: Modeling normal and pathological processes
Sharma, A., & Malik, A. (2026) · Elsevier eBooks · doi:10.1016/b978-0-443-45010-5.00007-1
Data Quality: Models, Measures, and Tools
Chen, H., 沙科伟, Zhou, Y., & Tu, F. (2026) · Springer handbooks · doi:10.1007/978-3-031-92372-2_55
Self-citation
Metadata
- Authors
- Chen, H., Chen, J., & Ding, J.
- Affiliations
- University of North Texas (US)
- Venue
- IEEE Transactions on Reliability
- Published
- 2021-06
- Volume
- 70
- Issue
- 2
- Pages
- 831-847
- ISSN
- 0018-9529 (ISSN-L)
- 1558-1721
- OpenAlex
- W3159579291
- Semantic Scholar
- 591160ceaa7d65ee9c65186b6907a1f82a419902
- Cited by
- 166
- References
- 58
- Citations by year
- 2026: 24 · 2025: 51 · 2024: 41 · 2023: 28 · 2022: 18 · 2021: 4
- FWCI
- 13.84
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- IEEE Transactions on Reliability (publishedVersion)
- Funders
- National Science Foundation (1852249)
- National Security Agency (H98230-20-1-0417)
- State Key Laboratory for Novel Software Technology in Nanjing University, China (KFKT2019A19)
- Topics
- Physical Sciences › Computer Science › Information Systems › Data Quality and Management
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- data quality, transfer learning, social media text, data validation, performance improvement, dataset quality, validation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.