Machine learning with big data: challenges and approaches.
Open accessThe DOI matches its Crossref record; 108 of the 109 references it cites were found in a registry or an index.
Abstract
The Big Data revolution promises to transform how we live, work, and think by enabling process optimization, empowering insight discovery and improving decision making. The realization of this grand potential relies on the ability to extract value from such massive data through data analytics; machine learning is at its core because of its ability to learn from data and provide data driven insights, decisions, and predictions. However, traditional machine learning approaches were developed in a different era, and thus are based upon multiple assumptions, such as the data set fitting entirely into memory, what unfortunately no longer holds true in this new context. These broken assumptions, together with the Big Data characteristics, are creating obstacles for the traditional techniques. Consequently, this paper compiles, summarizes, and organizes machine learning challenges with Big Data. In contrast to other research that discusses challenges, this work highlights the cause–effect relationship by organizing challenges according to Big Data Vs or dimensions that instigated the issue: volume, velocity, variety, or veracity. Moreover, emerging machine learning approaches and techniques are discussed in terms of how they are capable of handling the various challenges with the ultimate objective of helping practitioners select appropriate solutions for their use cases. Finally, a matrix relating the challenges and approaches is presented. Through this process, this paper provides a perspective on the domain, identifies research gaps and opportunities, and provides a strong foundation and encouragement for further research in the field of machine learning with Big Data.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“However, training these models requires large, diverse, and well-labeled datasets [94].”
From A Review of Passenger Counting in Public Transport Concepts with Solution Proposal Based on Image Processing and Machine Learning (2024) · cites this paper as [94]
In the citing paper
“it is also shifting from a single domain (e.g., domain expertise, or expert systems) to a cross-disciplinary approach through collaboration”
From Challenges and Future Directions of Big Data and Artificial Intelligence in Education (2020) · cites this paper as L’heureux et al., 2017
Citations
Cites
Data Mining: Practical Machine Learning Tools and Techniques
(2011) · Elsevier eBooks · doi:10.1016/c2009-0-19715-5 · Cited by 25,437
Regularization Paths for Generalized Linear Models via Coordinate Descent
Friedman, J., Hastie, T., & Tibshirani, R. (2010) · Journal of Statistical Software · doi:10.18637/jss.v033.i01 · Cited by 19,927 · cited 1×
“• Regularization paths [92] for linear models supports linear regression, two-class logistic regression, and multinomial regression problems.”
MethodThe WEKA data mining software: an update
Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., & Witten, I. H. (2009) · ACM SIGKDD Explorations Newsletter · doi:10.1145/1656274.1656278 · Cited by 17,750 · cited 1×
“However, at the moment, they support a limited number of algorithms compared to the R language [73], MATLAB [74], or Weka [75].”
MethodRepresentation Learning: A Review and New Perspectives
Bengio, Y., Courville, A., & Vincent, P. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.50 · Cited by 13,001 · cited 1×
“Representation learning is also often referred to as feature learning [100].”
BackgroundEnsemble methods in machine learning
Dietterich, T. G. (2000) · Lecture Notes in Computer Science · doi:10.1007/3-540-45014-9_1 · Cited by 8,408
(2011) · Cited by 5,203
Multiple comparisons among means
Dunn, O. J. (1961) · Journal of the American Statistical Association · doi:10.1080/01621459.1961.10482090 · Cited by 5,006 · cited 1×
“It suggests that if testing m hypotheses with a desired significance of α, each individual hypothesis should be tested at a significance level of α/m [43].”
BackgroundResilient distributed datasets: a fault-tolerant abstraction for in-memory cluster computing
Zaharia, M., Chowdhury, M., Das, T., Dave, A., Ma, J. H., McCauley, M., Franklin, M. J. M., Shenker, S. J., & Stoica, I. G. (2012) · UC Berkeley · Cited by 4,891
The Hadoop Distributed File System
Shvachko, K., Kuang, H., Radia, S., & Chansler, R. (2010) · 2010 IEEE 26th Symposium on Mass Storage Systems and Technologies (MSST) · doi:10.1109/msst.2010.5496972 · Cited by 4,834 · cited 8×
“In these stages, independent of the category of manipulations, processes can be embedded to capture data provenance and therefore remedy provenance challenge; an example is the Reduce and Map Provenance (RAMP) developed for MapReduce as an extension for Hadoop [64].”
MethodInfluentialHan, S., Mao, H., & Dally, W. J. (2015) · arXiv · doi:10.48550/arxiv.1510.00149 · Cited by 4,662 · cited 1×
“Deep compression has been proposed as a way of speeding up processing without the loss of accuracy [105].”
Background
Research works printed but not linked: 20
- goodfellow, 2016, Deep Learning
- marz, 2014, Apache Storm
- rouse, 2011, Machine Learning Definition
- 2015, R A Language and Environment for Statistical Computing, 1
- 2014, IBM Watson Ecosystem Program
- wu, 2016, Google’s neural machine translation system Bridging the gap between human and machine translation
- 2016, Machine Learning American
- 2016, Google Cloud Machine Learning
- 2016, Matlab
- 2013, Billion Devices Will Wirelessly Connect to the Internet of Everything in 2020
- krikorian, 2010, Twitter by the Numbers Twitter
- 2016, H20
- 2016, Apache Mahout
- abadi, 2016, TensorFlow Large-Scale Machine Learning on Heterogeneous Systems
- langford, 2007, Vowpal wabbit online learning project
- rouse, 2009, Predictive Analytics Definition
- sukumar, 2014, Machine learning in the big data era: Are we there yet?, Proc 20th ACM SIGKDD Conf Knowl Discovery Data Mining Workshop Data Sci Social Good (KDD), 1
- 2016, GPU Applications Transforming Computational Research and Engineering
- torrey, 2010, Handbook of Research on Machine Learning Applications and Trends
- 2016, Apache Giraph
Cited by
Items 101–200 of 1,002
Lv, Z., Ding, H., Wang, L., & Zou, Q. (2020) · Neurocomputing · doi:10.1016/j.neucom.2020.09.056 · Cited by 78
Monitoring machine learning models: a categorization of challenges and methods
Schröder, T., & Schulz, M. (2022) · Data Science and Management · doi:10.1016/j.dsm.2022.07.004 · Cited by 94
Ugwu, L. I., Morgan, Y., & Ibrahim, H. (2021) · International Journal of Hydrogen Energy · doi:10.1016/j.ijhydene.2021.10.208 · Cited by 76
Chen, M., Claramunt, C., Çöltekin, A., Liu, X., Peng, P., Robinson, A. C., Wang, D., Strobl, J., Wilson, J. P., Batty, M., et al. (2023) · Earth-Science Reviews · doi:10.1016/j.earscirev.2023.104438 · Cited by 74
“Nonetheless, there is no doubt that big data creates challenges related to its characteristics (i.e., its dimensions) (Khan et al., 2017; L ’ heureux et al., 2017; Yang et al., 2017).”
BackgroundExploring the Intersection of Machine Learning and Big Data: A Survey
Dritsas, E., & Τρίγκα, Μ. (2025) · Machine Learning and Knowledge Extraction · doi:10.3390/make7010013 · Cited by 94
“[17] This paper compiles challenges associated with the use of ML and big data, categorizing them by the dimensions of big data (volume, velocity, variety, and veracity).”
Evaluating GPT and BERT models for protein–protein interaction identification in biomedical text
Rehana, H., Çam, N. B., Basmaci, M., Zheng, J., Jemiyo, C., He, Y. O., Özgür, A., & Hur, J. (2024) · Bioinformatics Advances · doi:10.1093/bioadv/vbae133 · Cited by 78
“Typically, best-performing traditional models require complex and time-consuming feature extraction processes that demand significant domain expertise (L’heureux et al. 2017, Elshawi et al. 2019).”
BackgroundZhu, X., Guo, H., & Huang, J. J. (2024) · Sustainable Cities and Society · doi:10.1016/j.scs.2024.105508 · Cited by 72
Requirements for Big Data Adoption for Railway Asset Management
Mcmahon, P., Zhang, T., & Dwight, R. A. (2020) · IEEE Access · doi:10.1109/access.2020.2967436 · Cited by 78
“In addition, big data has facilitated the use of techniques such as machine learning and expert systems for knowledge discovery [35].”
BackgroundHouse Prices Prediction with Machine Learning Algorithms
Fan, C., Cui, Z., & Zhong, X. (2018) · International Conference on Machine Learning and Computing (ICMLC) · doi:10.1145/3195106.3195133 · Cited by 70
“Meanwhile, machine learning algorithms [1] perform excellently and have been widely used in such problems [2-4].”
MethodŞentürk, Ş., Şentürk, F., & Karaca, H. (2023) · Journal of Cleaner Production · doi:10.1016/j.jclepro.2023.137096 · Cited by 69
Kunjiappan, S., Ramasamy, L. K., Suthendran, K., Pavadai, P., Theivendren, P., & Palanisamy, P. (2024) · Scientific Reports · doi:10.1038/s41598-023-49839-y · Cited by 69
“Machine learning is generally used to learn the underlying relationship between input and output responses, which is learned from a set of training data 43 .”
BackgroundDeep Learning: Edge-Cloud Data Analytics for IoT
Ghosh, A. M., & Grolinger, K. (2019) · 2019 IEEE Canadian Conference of Electrical and Computer Engineering (CCECE) · doi:10.1109/ccece.2019.8861806 · Cited by 62
“This can be explained by curse of dimensionality, the negative effect of a large number of features on accuracy of ML algorithms [8].”
BackgroundMethodInfluentialSelf-citationSurrogate Models in Rock and Soil Mechanics: Integrating Numerical Modeling and Machine Learning
Furtney, J., Thielsen, C., Fu, W., & Goc, R. L. (2022) · Rock Mechanics and Rock Engineering · doi:10.1007/s00603-021-02720-8 · Cited by 74
Can Artificial Intelligence Accelerate Fluid Mechanics Research?
Drikakis, D., & Sofos, F. D. (2023) · Fluids · doi:10.3390/fluids8070212 · Cited by 67
Of leaders and laggards - Towards digitalization of the process industries
Aaldering, L. J., & Song, C. H. (2021) · Technovation · doi:10.1016/j.technovation.2020.102211 · Cited by 66
Naphade, M., Wang, S., Anastasiu, D. C., Tang, Z., Chang, M., Yao, Y., Zheng, L., Rahman, M. S., Arya, M. S., Sharma, A., et al. (2023) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw59228.2023.00586 · Cited by 79
Truth from the machine: artificial intelligence and the materialization of identity
Keyes, O., Hitzig, Z., & Blell, M. (2021) · Interdisciplinary Science Reviews · doi:10.1080/03080188.2020.1840224 · Cited by 51
“Consequently, algorithmi-cally revealed correlations way may not actually be accurate (L ’ heureux et al. 2017).”
Enhancing Weather-Related Power Outage Prediction by Event Severity Classification
Yang, F., Watson, P. L., Koukoula, M., & Anagnostou, E. N. (2020) · IEEE Access · doi:10.1109/access.2020.2983159 · Cited by 68
“The sensitivity of ML models to their training datasets [30] results in biases and random errors in the predictions, which limits their accuracy.”
BackgroundHow to translate artificial intelligence? Myths and justifications in public discourse
Roberge, J., Senneville, M., & Morin, K. (2020) · Big Data & Society · doi:10.1177/2053951720919968 · Cited by 67
“…such subtleties are shaping new data architectures from the inside-out while also interacting and coping with data originating from the outside—social — world (L’Heureux et al., 2017), thus once again proving how critical it is to never lose sight of the contexts in which those systems operate.”
BackgroundLearning Medical Image Denoising with Deep Dynamic Residual Attention Network
Sharif, S. M. A., Naqvi, R. A., & Biswas, M. (2020) · Mathematics · doi:10.3390/math8122192 · Cited by 69
“However, to generalize the performance of any deep method on a spacious data space, a substantial amount of training data samples is mandatory [51].”
BackgroundSE-OnionNet: A Convolution Neural Network for Protein–Ligand Binding Affinity Prediction
Wang, S., Liu, D., Ding, M., Du, Z., Zhong, Y., Song, T., Zhu, J., & Zhao, R. (2021) · Frontiers in Genetics · doi:10.3389/fgene.2020.607824 · Cited by 64
“This is a laborious and complicated process, and it cannot be efficiently applied in machine learning methods, particularly for large-scale data (Lheureux et al., 2017).”
BackgroundDeep Learning for High-Impedance Fault Detection: Convolutional Autoencoders
Rai, K., Hojatpanah, F., Ajaei, F. B., & Grolinger, K. (2021) · Energies · doi:10.3390/en14123623 · Cited by 62
“Moreover, the existing technical literature on ML-based HIF detection relies on features extracted by resource intensive signal processing and data transformation techniques [28].”
BackgroundSelf-citationRahul, K., Banyal, R. K., & Arora, N. K. (2023) · Journal Of Big Data · doi:10.1186/s40537-023-00808-2 · Cited by 73
“Some factors affect verintegrityuch as data provenance, uncertainty, and dirty and noisy data [7].”
BackgroundMethodInfluentialSun, Z., He, T., Ren, Z., Wang, C., Liu, X., Zhang, Z., Zhou, J., Guo, X., Yang, Y., & Lee, C. (2025) · SmartSys · doi:10.1002/sys3.4 · Cited by 65
“However, a good machine learning model is often accompanied by a large amount of data and high computational complexity [52, 53], and cloud computing has become a trending solution to reduce the computing burden of terminals under wearable/portable application scenarios thanks to the rapid…”
Bloom filter based optimization scheme for massive data handling in IoT environment
Singh, A., Garg, S., Batra, S., Kumar, N., & Rodrigues, J. J. P. C. (2017) · Future Generation Computer Systems · doi:10.1016/j.future.2017.12.016 · Cited by 64
Machine learning in geography–Past, present, and future
Lavallin, A. V., & Downs, J. A. (2021) · Geography Compass · doi:10.1111/gec3.12563 · Cited by 69
Naphade, M., Wang, S., Anastasiu, D. C., Tang, Z., Chang, M., Yao, Y., Zheng, L., Rahman, M. S., Venkatachalapathy, A., Sharma, A., et al. (2022) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw56347.2022.00378 · Cited by 67
Karwehl, L. J., & Kauffeld, S. (2021) · Gruppe Interaktion Organisation Zeitschrift für Angewandte Organisationspsychologie (GIO) · doi:10.1007/s11612-021-00548-y · Cited by 75
Kotenko, I. V., Евгеньевич, И. К., & Buinevich, M. (2022) · Sensors · doi:10.3390/s22041335 · Cited by 60
“According to both general theory and a large number of scientific papers and their reviews ([79,80]), all ML problems can be divided into the following.”
BackgroundImportance of machine learning for enhancing ecological studies using information-rich imagery
Dujon, A. M., & Schofield, G. (2019) · Endangered Species Research · doi:10.3354/esr00958 · Cited by 66
Feature Selection for Phishing Website Classification
Shabudin, S., Samsiah, N., Ariffin, K. A. Z., & Aliff, M. (2020) · International Journal of Advanced Computer Science and Applications · doi:10.14569/ijacsa.2020.0110477 · Cited by 75
“In general, machine learning studies the research and algorithms construction that can learn from, and derive predictions about, data [24,25].”
Real-time End-to-End Federated Learning: An Automotive Case Study
Zhang, H., Bosch, J., & Olsson, H. H. (2021) · 2021 IEEE 45th Annual Computers, Software, and Applications Conference (COMPSAC) · doi:10.1109/compsac51774.2021.00070 · Cited by 49
Cheng, L., Yu, T., Jiang, H., Shi, S., Tan, Z., & Zhang, Z. (2019) · IEEE Access · doi:10.1109/access.2019.2897712 · Cited by 61
“, machine learning methods [54], [55]) that we have developed to effectively manage and control energy flow into the software and hardware systems of energy internet AE to match different application scenarios, such that realizing the interconnection of energy flow and information flow in the energy internet system in…”
MethodAhmad, I., Shahabuddin, S., Sauter, T., Harjula, E., Kumar, T., Meisel, M., Juntti, M. J., & Ylianttila, M. (2020) · IEEE Industrial Electronics Magazine · doi:10.1109/mie.2020.2979272 · Cited by 64
“Be cause the tools and algorithms of AI require data, and a common assump tion is that the more there data are, the better the results will be [11], it is highly likely that the needed resourc es are not available locally or in the edge.”
BackgroundMethodMachine Learning Research Trends in Africa: A 30 Years Overview with Bibliometric Analysis Review
Ezugwu, A. E.-S., Oyelade, O. N., Ikotun, A. M., Agushaka, J. O., & Ho, Y. (2023) · Archives of Computational Methods in Engineering · doi:10.1007/s11831-023-09930-z · Cited by 67
Ellis, C., Jacobs, M., & Kendall, D. L. (2021) · American Journal of Speech-Language Pathology · doi:10.1044/2021_ajslp-20-00346 · Cited by 57
Organizational Implementation of AI: C raft and M echanical W ork
Hopf, K., Müller, O., Shollo, A., & Thiess, T. (2023) · California Management Review · doi:10.1177/00081256231197445 · Cited by 57
“47”
Production Coordinator, P. G. O. I. (. G., Roy, S., Hasan, M., & Product Developer, G.-E. G. (2021) · doi:10.63125/t5dcb097 · Cited by 56
Evolutionary feature selection on high dimensional data using a search space reduction approach
García-Torres, M., Ruíz, R., & Divina, F. (2022) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2022.105556 · Cited by 60
The Democratization of Artificial Intelligence: Net Politics in the Era of Learning Algorithms
Sudmann, A. (2019) · AI critique · doi:10.14361/9783839447192 · Cited by 59
Kittichotsatsawat, Y., Tippayawong, N., & Tippayawong, K. Y. (2022) · Scientific Reports · doi:10.1038/s41598-022-18635-5 · Cited by 54
Enabling AutoML for Zero-Touch Network Security: Use-Case Driven Analysis
Yang, L., Rajab, M. E., Shami, A. A., & Muhaidat, S. (2024) · IEEE Transactions on Network and Service Management · doi:10.1109/tnsm.2024.3376631 · Cited by 55
“3) Feature Extraction: It aims to reduce dimensionality by applying mapping functions, creating a more concise representation of the original dataset while potentially increasing model learning efficiency [50].”
5G Wireless Networks Meet Big Data Challenges, Trends, and Applications
Zhu, C., Cheng, X., Ye, H., Yang, J., Xu, L., & Chao, K. (2019) · IEEE SmartWorld, Ubiquitous Intelligence & Computing, Advanced & Trusted Computing, Scalable Computing & Communications, Cloud & Big Data Computing, Internet of People and Smart City Innovation (SmartWorld/SCALCOM/UIC/ATC/CBDCom/IOP/SCI) · doi:10.1109/smartworld-uic-atc-scalcom-iop-sci.2019.00272 · Cited by 56
“Depend on data analytics, it is expected to learn from large amounts of data, discover hidden knowledge and provide better decision making methods [3].”
BackgroundSWiBluX: Multi-Sensor Deep Learning Fingerprint for Precise Real-Time Indoor Tracking
Belmonte-Hernandez, A., Hernandez-Penaloza, G., Martin Gutierrez, D., & Alvarez, F. (2019) · IEEE Sensors Journal · doi:10.1109/jsen.2019.2892590 · Cited by 53
Taxonomy of Machine Learning Safety: A Survey and Primer
Mohseni, S., Wang, H., Xiao, C., Yu, Z., Wang, Z., & Yadawa, J. (2023) · ACM Computing Surveys · doi:10.1145/3551385 · Cited by 53
“The quality of training data is an important factor for machine learning with big data [150].”
BackgroundImplementing Feature and Metric Stores for Machine Learning Models in the Gaming Industry
Paulraj, B. (2021) · European Economic Letters (EEL) · Cited by 53
Silva, N. M., Barros, J., Santos, M. Y., Costa, C., Cortez, P. C., Carvalho, M. S., & Gonçalves, J. N. C. (2021) · Electronics · doi:10.3390/electronics10182221 · Cited by 52
“For machine learning tasks, we can use Spark, H2O, or TensorFlow [32].”
MethodThe decay of Six Sigma and the rise of Quality 4.0 in manufacturing innovation
Escobar, C. A., Arregoyta, D. M., & Morales-Menéndez, R. (2023) · Quality Engineering · doi:10.1080/08982112.2023.2206679 · Cited by 52
Entity Linking: An Issue to Extract Corresponding Entity With Knowledge Base
Wu, G., He, Y., & Hu, X. (2018) · IEEE Access · doi:10.1109/access.2017.2787787 · Cited by 52
“Big data has become a symbol of the era [1]–[3].”
BackgroundA Comprehensive Survey on Big Data Analytics: Characteristics, Tools and Techniques
Shahnawaz, M., & Kumar, M. (2025) · ACM Computing Surveys · doi:10.1145/3718364 · Cited by 68
“The variety of Big Data encompasses its structure, data types, semantic interpretation, and sources, presenting challenges that significantly affect data analysis [111].”
InfluentialA critical review of the role of technology and context in digital health research
Nielsen, P., & Sahay, S. (2022) · Digital Health · doi:10.1177/20552076221109554 · Cited by 58
“Digital technologies, such as the internet of things, smartphones, social media, and digital platforms, also lead to the generation of new and diverse forms of data, such as big data and biodata, and opportunities to do new ways of analytics, such as those based on machine learning (see, for example, L’Heureux et al.(…”
BackgroundA Data-Driven Health Prognostics Approach for Steam Turbines Based on Xgboost and DTW
Que, Z., & Xu, Z. (2019) · IEEE Access · doi:10.1109/access.2019.2927488 · Cited by 50
“The data-driven approaches are to make use of large numbers of historical data to enable knowledge discovery and precise decision making [11].”
BackgroundProcess-independent construction stage analysis of self-anchored suspension bridges
Wang, X., Wang, H., Sun, Y., Mao, X., & Tang, S. (2020) · Automation in Construction · doi:10.1016/j.autcon.2020.103227 · Cited by 50
Traumatic Brain Injury (TBI) Detection: Past, Present, and Future
Alouani, A. T., & Elfouly, T. (2022) · Biomedicines · doi:10.3390/biomedicines10102472 · Cited by 49
“To take advantage of such data, machine learning was one of the artificial intelligence technologies used [48,50,51].”
MethodSoybean yield prediction by machine learning and climate
Torsoni, G. B., Aparecido, L. E. D. O., dos Santos, G. M., Chiquitto, A. G., de Moraes, J. R. D. S. C., & Rolim, G. D. S. (2023) · Theoretical and Applied Climatology · doi:10.1007/s00704-022-04341-9 · Cited by 52
“Several methodologies can be used to overcome these challenges, including a series of adaptations for the correct use of ML in Big Data systems, mainly in agriculture (L’Heureux et al. 2017).”
MethodGuo, W., Qureshi, N. M. F., Jarwar, M. A., Kim, J., & Shin, D. R. (2023) · Sustainable Energy Technologies and Assessments · doi:10.1016/j.seta.2022.102990 · Cited by 60
Shome, D., Waqar, O., & Khan, W. U. (2022) · Transactions on Emerging Telecommunications Technologies · doi:10.1002/ett.4458 · Cited by 51
“As the datasets and models have been growing at an unprecedented rate, it is becoming impractical to train a model on a single machine [29].”
BackgroundIntelligent Traffic Management in Next-Generation Networks
Aouedi, O., Piamrat, K., & Parrein, B. (2022) · Future Internet · doi:10.3390/fi14020044 · Cited by 56
“…dataset can contain irrelevant (a feature that provides no useful information) and redundant (a feature with predictive ability covered by another) features, which cause an extra computational cost for both storage and processing in an SDN environment and decreases the model’s performance [51,52].”
BackgroundLi, Q., Kamaruddin, N., Yuhaniz, S. S., & Al-Jaifi, H. A. A. (2024) · Scientific Reports · doi:10.1038/s41598-023-50783-0 · Cited by 48
An Empirical Comparison of Supervised Machine Learning Algorithms for Internet of Things Data
Khadse, V., Mahalle, P. N., & Biraris, S. V. (2018) · 2018 Fourth International Conference on Computing Communication Control and Automation (ICCUBEA) · doi:10.1109/iccubea.2018.8697476 · Cited by 39
Enhancing Social Media Platforms with Machine Learning Algorithms and Neural Networks
Taherdoost, H. (2023) · Algorithms · doi:10.3390/a16060271 · Cited by 56
Unlocking the power of machine learning in big data: a scoping survey
Surur, F. M., Mamo, A. A., Gebresilassie, B. G., Mekonen, K. A., Golda, A., Behera, R. K., & Kumar, K. S. K. (2025) · Data Science and Management · doi:10.1016/j.dsm.2025.02.004 · Cited by 64
Autonomous road vehicles: recent issues and expectations
Skrickij, V., Šabanovič, E., & Žuraulis, V. (2020) · IET Intelligent Transport Systems · doi:10.1049/iet-its.2018.5513 · Cited by 53
“Such algorithms proliferate because of the emergence of extensive data, high-performance processing hardware and software technologies [51, 52].”
BackgroundConsiderations for a more ethical approach to data in AI: On data representation and infrastructure
Baird, A., & Schuller, B. (2020) · Frontiers in Big Data · doi:10.3389/fdata.2020.00025 · Cited by 46
“The process of managing what Schembera and Durán (2020), describes as “tangible data” can be extremely time-consuming and costly for those involved and human or machine error can propagate, resulting in biases or leading to mostly unusable data (L’heureux et al., 2017).”
BackgroundPapachristou, N., Kotronoulas, G., Δικαίος, Ν., Allison, S. J., Eleftherochorinou, H., Rai, T., Kunz, H. R., Barnaghi, P. M., Miaskowski, C. A., & Bamidis, P. D. (2023) · Seminars in Oncology Nursing · doi:10.1016/j.soncn.2023.151433 · Cited by 58
“Another challenge is the various types of data drift that can occur, such as concept drift 109,110 and covariate shift.”
Artificial Intelligence: An Overview
Jaboob, A. S., Durrah, O., & Chakir, A. (2024) · Synthesis lectures on engineering, science, and technology · doi:10.1007/978-3-031-50300-9_1 · Cited by 70
Hussain, F. A., Hussain, R., Anpalagan, A., & Benslimane, A. (2020) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2020.2965796 · Cited by 52
Hopf, K., & Reifenrath, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.12140 · Cited by 53
“Besides, we use two problems from the area of small datasets (L’Heureux et al., 2017; Bosu and MacDonell, 2013), namely (III) few observations with many features and (IV) few observations of a minority class (i.e., imbalanced classes).”
MethodCyberbullying Detection Based on Emotion
Al-Hashedi, M., Soon, L.-K., Goh, H.-N., Lim, A. H. L., & Siew, E.-G. (2023) · IEEE Access · doi:10.1109/access.2023.3280556 · Cited by 45
“The effectiveness of machine learning models is heavily dependent on the quality of the training datasets [15], [16].”
BackgroundShi, J.-J., Zhang, W., Wang, W., Sun, Y., Xu, C.-Y., Zhu, H., & Sun, Z. (2021) · Engineering Geology · doi:10.1016/j.enggeo.2021.106235 · Cited by 46
Health insurance fraud detection based on multi-channel heterogeneous graph structure learning
Hong, B., Lu, P., Xu, H., Lu, J., Lin, K.-B., & Yang, F. (2024) · Heliyon · doi:10.1016/j.heliyon.2024.e30045 · Cited by 57
“During the health insurance fraud detection process, traditional machine learning methods often struggle with sparse data and difficult feature extraction, making it challenging to leverage the heterogeneity of the data [14,15].”
A novel wide & deep transfer learning stacked GRU framework for network intrusion detection
Singh, N. B., Singh, M. M., Sarkar, A., & Mandal, J. K. (2021) · Journal of Information Security and Applications · doi:10.1016/j.jisa.2021.102899 · Cited by 75
A Survey of Machine Learning for Big Data Processing
Almutiri, R., Alhabeeb, S., Alhumud, S., & Khan, R. U. (2022) · Journal on big data · doi:10.32604/jbd.2022.028363 · Cited by 44
Graph neural networks for single-cell omics data: a review of approaches and applications
Li, S., Hua, H., & Chen, S. (2025) · Briefings in Bioinformatics · doi:10.1093/bib/bbaf109 · Cited by 43
“It does not have a fixed structure or linear order, and its relationships are defined by the graph’s topology. alters the fundamental assumption of traditional machine learning algorithms that the random variables are independent [ 49].”
Adam, M., Albaseer, A. M., Baroudi, U. A., & Abdallah, M. M. (2025) · IEEE Open Journal of the Communications Society · doi:10.1109/ojcoms.2025.3554537 · Cited by 61
On Smart Geometric Non-Destructive Evaluation: Inspection Methods, Overview, and Challenges
Jaber, A., Karganroudi, S. S., Meiabadi, M. S., Aminzadeh, A., Ibrahim, H., Adda, M., & Taheri, H. (2022) · Materials · doi:10.3390/ma15207187 · Cited by 42
Federated Learning Systems: Architecture Alternatives
Zhang, H., Bosch, J., & Olsson, H. H. (2020) · Asia-Pacific Software Engineering Conference (APSEC) · doi:10.1109/apsec51365.2020.00047 · Cited by 57
“Also, our previous research shows that the transition from prototype to the production-quality deployment of ML models proves to be challenging for many companies [8] [9].”
BackgroundAouedi, O., Piamrat, K., Hamma, S., & Perera, J. K. M. (2021) · Annals of Telecommunications · doi:10.1007/s12243-021-00889-1 · Cited by 54
“, features) of data, the predictive ability of an algorithm decreases as well as causing an extra computational cost for both storage and processing [8].”
BackgroundGoliatt, L., Saporetti, C. M., & Pereira, E. (2023) · Fuel · doi:10.1016/j.fuel.2023.128682 · Cited by 42
Singh, P., Kaur, A., Batth, R. S., Kaur, S., & Gianini, G. (2021) · Neural Computing and Applications · doi:10.1007/s00521-021-05798-x · Cited by 42
“For overviews of this area of research, one can see Sughasiny et al. [63] and Alexandra et al. [41].”
BackgroundTanui, C. K., Karanth, S., Njage, P. M. K., Meng, J., & Pradhan, A. K. (2021) · LWT · doi:10.1016/j.lwt.2021.112701 · Cited by 41
Recent Advances in Data Engineering for Networking
Zeydan, E., & Mangues‐Bafalluy, J. (2022) · IEEE Access · doi:10.1109/access.2022.3162863 · Cited by 64
“(xviii) Limited data availability: Unavailability of sufficient data for AI/ML training and model building purposes is a major challenge in almost all industrial use cases [223].”
BackgroundApplication of machine learning in chemical engineering: outlook and perspectives
Sharah, A. A., Owida, H. A., Alnaimat, F., & Abuowaida, S. F. A. (2023) · IAES International Journal of Artificial Intelligence · doi:10.11591/ijai.v13.i1.pp619-630 · Cited by 45
“Anomaly detection methods based on self-learning unsupervised neural networks have recently been developed [83], [91], [114], [115].”
MethodApplicable and generalizable machine learning for intelligent welding in automotive manufacturing
Wang, P. E., Ghassemi-Armaki, H., Pour, M., Zhao, X., Ma, J., Sattari, K., & Carlson, B. E. (2025) · Welding in the World · doi:10.1007/s40194-025-01951-5 · Cited by 44
“ML models, determined by their data-driven learning nature, have a strong dependence on training data and are prone to low fidelity if training data have unbalanced or uncomprehensive coverage of application scenarios [26, 27].”
Advanced Techniques for Predicting the Future Progression of Type 2 Diabetes
Islam, M. S., Qaraqe, M., Belhouari, S. B., & Abdul‐Ghani, M. A. (2020) · IEEE Access · doi:10.1109/access.2020.3005540 · Cited by 43
“classifiers work best when the data are balanced [20].”
BackgroundDigitalization in microbiology – Paving the path to sustainable circular bioeconomy
Krüger, A., Schäfers, C., Busch, P., & Antranikian, G. (2020) · New Biotechnology · doi:10.1016/j.nbt.2020.06.004 · Cited by 42
Hatamzad, M., Polanco, G., & Casselgren, J. (2022) · Knowledge-Based Systems · doi:10.1016/j.knosys.2022.108682 · Cited by 42
A new hybrid deep learning model for monthly oil prices forecasting
Guan, K., & Gong, X. (2023) · Energy Economics · doi:10.1016/j.eneco.2023.107136 · Cited by 45
Predict Stock Market Behavior: Role of Machine Learning Algorithms
Gurav, U., & Sidnal, N. S. (2018) · Advances in intelligent systems and computing · doi:10.1007/978-981-10-7245-1_38 · Cited by 42
Gamma ray log generation from drilling parameters using deep learning
Osarogiagbon, A. U., Oloruntobi, O., Khan, F. I., Venkatesan, R., & Butt, S. D. (2020) · Journal of Petroleum Science and Engineering · doi:10.1016/j.petrol.2020.107906 · Cited by 39
Rizvi, S. M. H., Sadanandan, S. K., & Srivastava, A. K. (2021) · IEEE Access · doi:10.1109/access.2021.3107248 · Cited by 39
“This phenomenon in ML literature is known as concept drift [31].”
BackgroundChen, X., & Geyer, P. (2021) · Applied Energy · doi:10.1016/j.apenergy.2021.118240 · Cited by 44
Tor Traffic Classification from Raw Packet Header using Convolutional Neural Network
Kim, M., & Anpalagan, A. (2018) · IEEE International Conference on Knowledge Innovation and Invention (ICKII) · doi:10.1109/ickii.2018.8569113 · Cited by 48
“2, etc, neural networks can learn and classify these packets by itself without our hand-design feature extraction or engineering which is the process of creating features to make the ML perform better [12].”
BackgroundKim, Y.-M., Jung, D., Chang, Y.-E., & Choi, D. (2019) · Electronics · doi:10.3390/electronics8040468 · Cited by 42
“The big data revolution promises to transform how we live, work, and think by employing optimization processes, empowering insight discovery, and improving decision-making [4].”
BackgroundUse of Machine Learning algorithms for designing efficient cyber security solutions
Soni, S., & Bhushan, B. (2019) · International Conference on Intelligent Computing, Instrumentation and Control Technologies (ICICICT) · doi:10.1109/icicict46008.2019.8993253 · Cited by 38
“It is because the ML techniques required the data to learn so it is important to have good understanding about the data in order to design a model or apply different algorithms [8, 9].”
MethodKamm, S., Jazdi, N., & Weyrich, M. (2021) · Procedia CIRP · doi:10.1016/j.procir.2021.11.164 · Cited by 43
Network analysis in a peer-to-peer energy trading model using blockchain and machine learning
Shukla, S., Hussain, S., Irshad, R. R., Alattab, A. A., Thakur, S., Breslin, J. G., Hassan, M. F., Abimannan, S., Husain, M. S., & Jameel, S. M. (2023) · Computer Standards & Interfaces · doi:10.1016/j.csi.2023.103799 · Cited by 41
Alfarraj, O., AlZubi, A. A., & Tolba, A. K. R. (2018) · Neural Computing and Applications · doi:10.1007/s00521-018-3612-0 · Cited by 37
RetractedShabestari, F., Rahmani, A. M., Navimipour, N. J., & Jabbehdari, S. (2018) · Journal of Network and Computer Applications · doi:10.1016/j.jnca.2018.11.007 · Cited by 37
The Connectivity Evaluation Among Wells in Reservoir Utilizing Machine Learning Methods
Du, S., Wang, R., Wei, C., Wang, Y., Zhou, Y., Wang, J., & Song, H. (2020) · IEEE Access · doi:10.1109/access.2020.2976910 · Cited by 37
Metadata
- Authors
- L'Heureux, A., Grolinger, K., Elyamany, H. F., & Capretz, M. A. M.
- Affiliations
- Western University (CA)
- Suez Canal University (EG)
- Venue
- IEEE Access
- Published
- 2017
- Volume
- 5
- Pages
- 7776-7797
- ISSN
- 2169-3536 (ISSN-L)
- OpenAlex
- W2609731728
- Cited by
- 1,023
- References
- 109
- Citations by year
- 2026: 50 · 2025: 175 · 2024: 128 · 2023: 145 · 2022: 130 · 2021: 138 · 2020: 105 · 2019: 88 · 2018: 53 · 2017: 11
- FWCI
- 112.99
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- IEEE Access (publishedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- NSERC CRD at Western University (477530-14)
- Topics
- Social Sciences › Decision Sciences › Information Systems and Management › Big Data Technologies and Applications
- Physical Sciences › Computer Science › Information Systems › Cloud Computing and Resource Management
- Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
- Keywords
- big data challenges, volume, velocity, varieties, honesty, scalable learning, data-driven decision making, research gaps
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.