An Automated Big Data Quality Anomaly Correction Framework Using Predictive Analysis
Open accessThe DOI matches its Crossref record; 30 of the 30 references it cites were found in a registry or an index.
Abstract
Big data has emerged as a fundamental component in various domains, enabling organizations to extract valuable insights and make informed decisions. However, ensuring data quality is crucial for effectively using big data. Thus, big data quality has been gaining more attention in recent years by researchers and practitioners due to its significant impact on decision-making processes. However, existing studies addressing data quality anomalies often have a limited scope, concentrating on specific aspects such as outliers or inconsistencies. Moreover, many approaches are context-specific, lacking a generic solution applicable across different domains. To the best of our knowledge, no existing framework currently automatically addresses quality anomalies comprehensively and generically, considering all aspects of data quality. To fill the gaps in the field, we propose a sophisticated framework that automatically corrects big data quality anomalies using an intelligent predictive model. The proposed framework comprehensively addresses the main aspects of data quality by considering six key quality dimensions: Accuracy, Completeness, Conformity, Uniqueness, Consistency, and Readability. Moreover, the framework is not correlated to a specific field and is designed to be applicable across various areas, offering a generic approach to address data quality anomalies. The proposed framework was implemented on two datasets and has achieved an accuracy of 98.22%. Moreover, the results have shown that the framework has allowed the data quality to be boosted to a great score, reaching 99%, with an improvement rate of up to 14.76% of the quality score.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Accelerating the XGBoost algorithm using GPU computing
Mitchell, R., & Frank, E. (2017) · PeerJ Computer Science · doi:10.7717/peerj-cs.127 · Cited by 359
Lian, B., Kartal, Y., Lewis, F. L., Mikulski, D., Hudas, G. R., Wan, Y., & Davoudi, A. A. (2022) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2022.3213526 · Cited by 66
Machine Learning Technologies for Big Data Analytics
Gandomi, A. H., Chen, F., & Abualigah, L. (2022) · Electronics · doi:10.3390/electronics11030421 · Cited by 60
Ali, S., Poulová, P., Yasmin, F., Danish, M., Akhtar, W., & Javed, H. M. U. (2020) · Journal of Open Innovation Technology Market and Complexity · doi:10.3390/joitmc6040190 · Cited by 59
An Advanced Big Data Quality Framework Based on Weighted Metrics
Elouataoui, W., Alaoui, I. E., Mendili, S. E., & Gahi, Y. (2022) · Big Data and Cognitive Computing · doi:10.3390/bdcc6040153 · Cited by 59
Self-citationContext-aware Big Data Quality Assessment: A Scoping Review
Fadlallah, H., Kilany, R., Dhayne, H., Haddad, R. E., Haque, R., Taher, Y., & Jaber, A. H. (2023) · Journal of Data and Information Quality · doi:10.1145/3603707 · Cited by 50
Novel Methods for Imputing Missing Values in Water Level Monitoring Data
Khampuengson, T., & Wang, W. (2023) · Water Resources Management · doi:10.1007/s11269-022-03408-6 · Cited by 36
Miss-gradient boosting regression tree: a novel approach to imputing water treatment data
Zhang, W., Li, R., Zhao, J., Wang, J., Meng, X., & Li, Q. (2023) · Applied Intelligence · doi:10.1007/s10489-023-04828-6 · Cited by 27
Perspective of anomaly detection in big data for data quality improvement
Keskar, V., Yadav, J., & Kumar, A. (2021) · Materials Today Proceedings · doi:10.1016/j.matpr.2021.05.597 · Cited by 23
A Secure Multi-User Database-as-a-Service Approach for Cloud Computing Privacy
Gahi, Y., & Alaoui, I. E. (2019) · Procedia Computer Science · doi:10.1016/j.procs.2019.11.006 · Cited by 20
Self-citation
Research works printed but not linked: 3
- (2023, November 09). Big Data Analytics: A Comparison of Tools and Applications|SpringerLink. Available online: https://link.springer.com/chapter/10.1007/978-3-319-74500-8_54. · DOI
- Goyle, K., Xie, Q., and Goyle, V. (2023). DataAssist: A Machine Learning Approach to Data Cleaning and Preparation. arXiv.
- (2023, July 08). Data Quality for Big Data and Machine Learning|Frontiers Research Topic. Available online: https://www.frontiersin.org/research-topics/35131/data-quality-for-big-data-and-machine-learning.
Cited by
Data Quality in the Age of AI: A Review of Governance, Ethics, and the FAIR Principles
Guillen-Aguinaga, M., Aguinaga-Ontoso, E., Guillen-Aguinaga, L., Guillen-Grima, F., & Aguinaga-Ontoso, I. (2025) · Data · doi:10.3390/data10120201 · Cited by 37
“Predictive approaches, such as XGBoost, can correct anomalies across multiple dimensions (accuracy, completeness, conformity, uniqueness, consistency, and readability), improving composite Data Quality Index scores from 70% to over 90% in large-scale applications [219].”
Data quality and uncertainty issues in flood prediction: a systematic review
Yu, J., Li, Y., Huang, X., & Ye, X. (2025) · International Journal of Digital Earth · doi:10.1080/17538947.2025.2495738 · Cited by 30
“However, issues related to data quality and uncertainty have not been given sufficient attention (Dottori, Di Baldassarre, and Todini 2013; Elouataoui, El Mendili, and Gahi 2023).”
A Survey on Data Quality Dimensions and Tools for Machine Learning Invited Paper
Zhou, Y., Tu, F., Sha, K., Ding, J., & Chen, H. (2024) · IEEE International Conference on Artificial Intelligence Testing (AITest) · doi:10.1109/aitest62860.2024.00023 · Cited by 43
“The quality of training data impacts the accuracy, reliability, and interpretation of ML models’ results [56], [57].”
BackgroundHasan, M. N., Papel, M. S. I., Rasel, I. H., Akter, S., Aktar, M. K., Abedin, M. Z., & Mani, L. (2025) · Edelweiss Applied Science and Technology · doi:10.55214/2576-8484.v9i7.9142 · Cited by 22
Avital, N., Egel, I., Weinstock, I., & Malka, D. (2024) · Inventions · doi:10.3390/inventions9060113 · Cited by 17
Metadata
- Authors
- Elouataoui, W., El Mendili, S., & Gahi, Y.
- Affiliations
- Université Ibn-Tofail (MA)
- Laboratory of Engineering Sciences, National School of Applied Sciences, Ibn Tofail University, Kenitra 14000, Morocco
- Corresponding author
- Elouataoui, Widad
- Gahi, Youssef
- Venue
- Data
- Publisher
- MDPI AG
- Published
- 2023-12-01
- Volume
- 8
- Issue
- 12
- Pages
- 182
- ISSN
- 2306-5729 (ISSN-L)
- OpenAlex
- W4389261600
- Cited by
- 27
- References
- 29
- Citations by year
- 2026: 6 · 2025: 13 · 2024: 6 · 2023: 1
- FWCI
- 4.80
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Data (publishedVersion, cc-by)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Physical Sciences › Computer Science › Information Systems › Data Quality and Management
- Social Sciences › Decision Sciences › Information Systems and Management › Big Data Technologies and Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
- Keywords
- big data quality, automated data cleaning, predictive modeling, data quality dimensions, accuracy, completeness, conformity, uniqueness, consistency, readability
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: An Automated Big Data Quality Anomaly Correction Framework Using Predictive Analysis. https://app.cochat.ai/papers/10.3390/data8120182
- The paper
- Elouataoui, W., El Mendili, S., & Gahi, Y. (2023). An Automated Big Data Quality Anomaly Correction Framework Using Predictive Analysis. Data. https://doi.org/10.3390/data8120182
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.