Large-scale machine learning systems in real-world industrial settings: A review of challenges and solutions

Caffe: Convolutional Architecture for Fast Feature Embedding

Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., & Darrell, T. (2014) · Proceedings of the 22nd ACM international conference on Multimedia · doi:10.1145/2647868.2654889 · Cited by 11,077 · see citing works

Untitled work

(2004) · Cited by 5,198 · see citing works

Caffe: Convolutional Architecture for Fast Feature EmbeddingPreprint

Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., & Darrell, T. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1408.5093 · Cited by 4,749 · see citing works

Guidelines for snowballing in systematic literature studies and a replication in software engineering

Wohlin, C. (2014) · Proceedings of the 18th International Conference on Evaluation and Assessment in Software Engineering · doi:10.1145/2601248.2601268 · Cited by 4,234 · see citing works

Wide & Deep Learning for Recommender Systems

Cheng, H.-T., Koç, L., Harmsen, J., Shaked, T., Chandra, T., Aradhye, H., Anderson, G., Corrado, G. S., Chai, W., Ispir, M., et al. (2016) · Workshop on Deep Learning for Recommender Systems (DLRS) · doi:10.1145/2988450.2988454 · Cited by 3,547 · see citing works

DeepTest

Tian, Y., Pei, K., Jana, S., & Ray, B. (2018) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3180155.3180220 · Cited by 1,400 · see citing works

Software Engineering for Machine Learning: A Case Study

Amershi, S., Begel, A., Bird, C., DeLine, R. A., Gall, H. C., Kamar, E. S., Nagappan, N., Nushi, B., & Zimmermann, T. (2019) · IEEE/ACM International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP) · doi:10.1109/icse-seip.2019.00042 · Cited by 1,181 · see citing works

Recommended Steps for Thematic Synthesis in Software Engineering

Cruzes, D. S., & Dyba, T. (2011) · 2011 International Symposium on Empirical Software Engineering and Measurement · doi:10.1109/esem.2011.36 · Cited by 960 · see citing works

Proceedings of the Conference on Human Factors in Computing Systems

Moore, J. E. (1989) · ACM eBooks · Cited by 960 · see citing works

Machine Learning and Deep Learning frameworks and libraries for large-scale data mining: a survey

Nguyen, G., Dlugolinský, Š., Bobák, M., Tran, V., García, Á. L., Heredia, I., Malík, P., & Hluchý, L. (2019) · Artificial Intelligence Review · doi:10.1007/s10462-018-09679-z · Cited by 924 · see citing works

Untitled work

(2015) · neural information processing systems · Cited by 912 · see citing works

Machine Learning Testing: Survey, Landscapes and Horizons

Zhang, J. M., Harman, M., Ma, L., & Liu, Y. (2022) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2019.2962027 · Cited by 897 · see citing works

Proceedings of the 24th ACM International on Conference on Information and Knowledge Management

(2015) · doi:10.1145/2806416 · Cited by 860 · see citing works

Proceedings of the 22nd ACM international conference on Multimedia

(2014) · doi:10.1145/2647868 · Cited by 741 · see citing works

A Survey of Deep Learning: Platforms, Applications and Emerging Research Trends

Hatcher, W. G., & Yu, W. (2018) · IEEE Access · doi:10.1109/access.2018.2830661 · Cited by 654 · see citing works

Applied Machine Learning at Facebook: A Datacenter Infrastructure Perspective

Hazelwood, K., Bird, S., Brooks, D., Chintala, S., Diril, U., Dzhulgakov, D., Fawzy, M., Jia, B., Jia, Y., Kalro, A., et al. (2018) · 2018 IEEE International Symposium on High Performance Computer Architecture (HPCA) · doi:10.1109/hpca.2018.00059 · Cited by 590 · see citing works

Proceedings of the 2014 ACM SIGMOD International Conference on Management of Data

(2014) · doi:10.1145/2588555 · Cited by 542 · see citing works

Proceedings of the 36th International Conference on Software Engineering

(2014) · doi:10.1145/2568225 · Cited by 477 · see citing works

A Survey of Machine Learning Techniques Applied to Self-Organizing Cellular Networks

Klaine, P. V., Imran, M. A., Onireti, O., & Souza, R. D. (2017) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2017.2727878 · Cited by 474 · see citing works

A systematic review and taxonomy of explanations in decision support and recommender systems

Nunes, I., & Jannach, D. (2017) · User Modeling and User-Adapted Interaction · doi:10.1007/s11257-017-9195-0 · Cited by 416 · see citing works

UX Design Innovation: Challenges for Working with Machine Learning as a Design Material

Dove, G., Halskov, K., Forlizzi, J., & Zimmerman, J. (2017) · Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3025739 · Cited by 391 · see citing works

The Architectural Implications of Autonomous Driving

Lin, S.-C., Zhang, Y., Hsu, C.-H., Skach, M., Haque, M. E., Tang, L., & Mars, J. (2018) · International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS) · doi:10.1145/3173162.3173191 · Cited by 383 · see citing works

Guest Editors' Introduction: On Applied Research in Machine Learning

Provost, F. J., & Kohavi, R. (1998) · Machine Learning · doi:10.1023/a:1007442505281 · Cited by 377 · see citing works

TFX: A TensorFlow-Based Production-Scale Machine Learning Platform

Baylor, D., Breck, E., Cheng, H.-T., Fiedel, N., Foo, C. Y., Haque, Z., Haykal, S., Ispir, M., Jain, V., Koc, L., et al. (2017) · Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/3097983.3098021 · Cited by 350 · see citing works

DeepTest: Automated Testing of Deep-Neural-Network-driven Autonomous CarsPreprint

Tian, Y., Pei, K., Jana, S., & Ray, B. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1708.08559 · Cited by 259 · see citing works

Large-scale machine learning at twitter

Lin, J., & Kolcz, A. (2012) · ACM SIGMOD International Conference on Management of Data (SIGMOD) · doi:10.1145/2213836.2213958 · Cited by 221 · see citing works

Machine Learning: The High Interest Credit Card of Technical Debt

Sculley, D., Holt, G. D., Golovin, D., Davydov, E. V., Phillips, T., Ebner, D., Chaudhary, V., & Young, M. (2014) · Cited by 218 · see citing works

A Taxonomy of Software Engineering Challenges for Machine Learning Systems: An Empirical Investigation

Lwakatare, L. E., Raj, A., Bosch, J., Olsson, H. H., & Crnkovic, I. (2019) · Lecture Notes in Business Information Processing · doi:10.1007/978-3-030-19034-7_14 · Cited by 214 · see citing works

The need for multivocal literature reviews in software engineering: complementing systematic literature reviews with grey literature

Garousi, V., Felderer, M., & Mäntylä, M. V. (2016) · Proceedings of the 20th International Conference on Evaluation and Assessment in Software Engineering · doi:10.1145/2915970.2916008 · Cited by 210 · see citing works

Proceedings of the 1st Workshop on Deep Learning for Recommender Systems

(2016) · doi:10.1145/2988450 · Cited by 209 · see citing works

Machine learning applied to software testing: A systematic mapping study

Durelli, V. H. S., Durelli, R. S., Borges, S. S., Endo, A. T., Eler, M. M., Dias, D. R. C., & Guimaraes, M. P. (2019) · IEEE Transactions on Reliability · doi:10.1109/tr.2019.2892517 · Cited by 207 · see citing works

Proceedings of the 1995 ACM SIGMOD international conference on Management of data - SIGMOD '95

(1995) · doi:10.1145/223784 · Cited by 204 · see citing works

Data scientists in software teams: State of the art and challenges

Kim, M., Zimmermann, T., DeLine, R., & Begel, A. (2018) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2017.2754374 · Cited by 199 · see citing works

Deep learning for self-driving cars

Rao, Q., & Frtunikj, J. (2018) · doi:10.1145/3194085.3194087 · Cited by 183 · see citing works

How does Machine Learning Change Software Development Practices?

Wan, Z., Xia, X., Lo, D., & Murphy, G. C. (2020) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2019.2937083 · Cited by 181 · see citing works

Model-Reuse Attacks on Deep Learning Systems

Ji, Y., Zhang, X., Ji, S., Luo, X., & Wang, T. (2018) · ACM SIGSAC Conference on Computer and Communications Security (CCS) · doi:10.1145/3243734.3243757 · Cited by 174 · see citing works

Deep learning in the automotive industry: Applications and tools

Luckow, A., Cook, M., Ashcraft, N., Weill, E., Djerekarov, E., & Vorster, B. (2016) · 2016 IEEE International Conference on Big Data (Big Data) · doi:10.1109/bigdata.2016.7841045 · Cited by 164 · see citing works

Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security

(2019) · doi:10.1145/3319535 · Cited by 163 · see citing works

Proceedings of the 14 International Symposium on High-Performance Computer Architecture

Fensch, C., & Cintra, M. S. (2008) · CiteSeer X (The Pennsylvania State University) · Cited by 159 · see citing works

M odel DB

Vartak, M., Subramanyam, H., Lee, W.-E., Viswanathan, S., Husnoo, S., Madden, S. R., & Zaharia, M. (2016) · Workshop on Human-In-the-Loop Data Analytics (HILDA) · doi:10.1145/2939502.2939516 · Cited by 157 · see citing works

Proceedings of the 19th International Conference on Evaluation and Assessment in Software Engineering

(2015) · doi:10.1145/2745802 · Cited by 153 · see citing works

Machine learning for targeted display advertising: transfer learning in action

Perlich, C., Dalessandro, B., Raeder, T., Stitelman, O., & Provost, F. J. (2013) · Machine Learning · doi:10.1007/s10994-013-5375-2 · Cited by 151 · see citing works

Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)

(2018) · doi:10.18653/v1/p18-2 · Cited by 122 · see citing works

Using Forward Snowballing to update Systematic Reviews in Software Engineering

Felizardo, K. R., Mendes, E., Kalinowski, M., de Souza, É. F., & Vijaykumar, N. L. (2016) · Proceedings of the ... ACM-IEEE International Symposium on Empirical Software Engineering and Measurement/International Symposium on Empirical Software Engineering and Measurement · doi:10.1145/2961111.2962630 · Cited by 110 · see citing works

Trials and tribulations of developers of intelligent systems: A field study

Hill, C., Bellamy, R., Erickson, T. D., & Burnett, M. (2016) · IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) · doi:10.1109/vlhcc.2016.7739680 · Cited by 108 · see citing works

How Do Engineers Perceive Difficulties in Engineering of Machine-Learning Systems? - Questionnaire Survey

Ishikawa, F., & Yoshioka, N. (2019) · doi:10.1109/cesser-ip.2019.00009 · Cited by 107 · see citing works

Ways of applying artificial intelligence in software engineering

FELDT, R. H., Neto, F. G. D. O., & Torkar, R. (2018) · International Workshop on Realizing Artificial Intelligence Synergies in Software Engineering (RAISE) · doi:10.1145/3194104.3194109 · Cited by 95 · see citing works

Proceedings of the 20th International Conference on Evaluation and Assessment in Software Engineering

Beecham, S., Kitchenham, B., & MacDonell, S. G. (2016) · doi:10.1145/2915970 · Cited by 93 · see citing works

The Architectural Implications of Autonomous Driving

Lin, S.-C., Zhang, Y., Hsu, C.-H., Skach, M., Haque, M. E., Tang, L., & Mars, J. (2018) · ACM SIGPLAN Notices · doi:10.1145/3296957.3173191 · Cited by 83 · see citing works

Novel Applications of Machine Learning in Software Testing

Briand, L. (2008) · doi:10.1109/qsic.2008.29 · Cited by 78 · see citing works

A Survey of Software Quality for Machine Learning Applications

Masuda, S., Ono, K., Yasue, T., & Hosokawa, N. (2018) · IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW) · doi:10.1109/icstw.2018.00061 · Cited by 71 · see citing works

Practical considerations in deploying AI for defect prediction

Tosun, A., Turhan, B., & Bener, A. (2009) · doi:10.1145/1540438.1540453 · Cited by 63 · see citing works

An Approach to Software Testing of Machine Learning Applications

Murphy, C., Kaiser, G. E., & Arias, M. (2007) · doi:10.7916/d8r49znr · Cited by 60 · see citing works

MODELDB: A System for Machine Learning Model Management.

Vartak, M. (2017) · Conference on Innovative Data Systems Research · Cited by 54 · see citing works

Learning in the “Real World”

Saitta, L., & Neri, F. (1998) · Machine Learning · doi:10.1023/a:1007448122119 · Cited by 53 · see citing works

A comparison of platforms for implementing and running very large scale machine learning algorithms

Cai, Z., Gao, Z. J., Luo, S., Perez, L. L., Vagena, Z., & Jermaine, C. (2014) · ACM SIGMOD International Conference on Management of Data (SIGMOD) · doi:10.1145/2588555.2593680 · Cited by 53 · see citing works

Agile Processes in Software Engineering and Extreme Programming

Kruchten, P. B., Fraser, S., & Coallier, F. (2019) · Lecture notes in business information processing · doi:10.1007/978-3-030-19034-7 · Cited by 46 · see citing works

A Comparison of Distributed Machine Learning Platforms

Zhang, K., Alqahtani, S., & Demirbaş, M. (2017) · International Conference on Computer Communication and Networks (ICCCN) · doi:10.1109/icccn.2017.8038464 · Cited by 45 · see citing works

It takes three to tango: Requirement, outcome/data, and AI driven development

Bosch, J., Olsson, H. H., & Crnković, I. (2018) · KTH Publication Database DiVA (KTH Royal Institute of Technology) · Cited by 38 · see citing works

What’s your ML test score? A rubric for ML production systems

Breck, E. J., Cai, S., Nielsen, E., Salib, M., & Sculley, D. (2016) · Cited by 33 · see citing works

DeepDIVA: A Highly-Functional Python Framework for Reproducible Experiments

Alberti, M., Pondenkandath, V., Würsch, M., Ingold, R., & Liwicki, M. (2018) · International Conference on Frontiers in Handwriting Recognition · doi:10.1109/icfhr-2018.2018.00080 · Cited by 32 · see citing works

A critical look at the relationship between AI and software engineering

Tsai, W., Heisler, K. G., Volovik, D., & Zualkernan, I. A. (2003) · IEEE Workshop on Languages for Automation · doi:10.1109/lfa.1988.24945 · Cited by 31 · see citing works

Continuous Integration of Machine Learning Models with ease.ml/ci: Towards a Rigorous Yet Practical TreatmentPreprint

Renggli, C., Karlaš, B., Ding, B., Liu, F., Schawinski, K., Wu, W., & Zhang, C. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.00278 · Cited by 30 · see citing works

On the Challenges of Translating NLP Research into Commercial Products

Dahlmeier, D. (2017) · Annual Meeting of the Association for Computational Linguistics (ACL) · doi:10.18653/v1/p17-2015 · Cited by 28 · see citing works

On Testing Data-Intensive Software Systems

Felderer, M., Russo, B., & Auer, F. (2019) · Springer eBooks · doi:10.1007/978-3-030-25312-7_6 · Cited by 26 · see citing works

Proceedings of the 5th International Conference on Predictor Models in Software Engineering

Ostrand, T. (2009) · Cited by 18 · see citing works

Proceedings of the Workshop on Human-In-the-Loop Data Analytics

(2016) · doi:10.1145/2939502 · Cited by 16 · see citing works

Practical Considerations of Deploying AI in Defect Prediction: A Case Study within the Turkish Telecommunication Industry

Tosun, A., Turhan, B., & Bener, A. (2009) · NPARC · Cited by 8 · see citing works

A system-level perspective to understand the vulnerability of deep learning systems

Liu, T., Xu, N., Liu, Q., Wang, Y., & Wen, W. (2019) · Asia and South Pacific Design Automation Conference (ASP-DAC) · doi:10.1145/3287624.3288751 · Cited by 6 · see citing works

Ease the Process of Machine Learning with Dataflow

Guo, T., Xu, J., Yan, X., Hou, J., Li, P., Li, Z., Guo, J., & Cheng, X. (2016) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/2983323.2983327 · Cited by 5 · see citing works

Machine learning in complex systems

Axtell, T. W., Overbey, L. A., & Woerner, L. (2018) · doi:10.1117/12.2309547 · Cited by 4 · see citing works

Modern computing: Vision and challenges

Gill, S. S., Wu, H., Patros, P., Ottaviani, C., Arora, P., Pujol, V. C., Haunschild, D., Parlikad, A. K., Cetinkaya, O., Lutfiyya, H., et al. (2024) · Telematics and Informatics Reports · doi:10.1016/j.teler.2024.100116 · Cited by 197 · see citing works

A software engineering perspective on engineering machine learning systems: State of the art and challenges

Giray, G. (2021) · Journal of Systems and Software · doi:10.1016/j.jss.2021.111031 · Cited by 187 · see citing works

Collaboration challenges in building ML-enabled systems

Nahar, N., Zhou, S., Lewis, G. A., & Kästner, C. (2022) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3510003.3510209 · Cited by 164 · see citing works

Artificial Intelligence in Biomaterials: A Comprehensive Review

Gokcekuyu, Y., Ekinci, F., Güzel, M. S., Açıcı, K., Aydin, S., & Aşuroğlu, T. (2024) · Applied Sciences · doi:10.3390/app14156590 · Cited by 126 · see citing works

Requirements engineering for artificial intelligence systems: A systematic mapping study

Ahmad, K., Abdelrazek, M., Arora, C., Bano, M., & Grundy, J. (2023) · Information and Software Technology · doi:10.1016/j.infsof.2023.107176 · Cited by 132 · see citing works

Systematic review of data-centric approaches in artificial intelligence and machine learning

Singh, P. (2023) · Data Science and Management · doi:10.1016/j.dsm.2023.06.001 · Cited by 124 · see citing works

On Continuous Integration / Continuous Delivery for Automated Deployment of Machine Learning Models using MLOps

Garg, S., Pundir, P., Rathee, G., Gupta, P. K., Garg, S., & Ahlawat, S. (2021) · IEEE International Conference on Artificial Intelligence and Knowledge Engineering (AIKE) · doi:10.1109/aike52691.2021.00010 · Cited by 99 · see citing works

Review of machine learning approaches for predicting mechanical behavior of composite materials

Sharma, H., Arora, G., Singh, M. K., Ayyappan, V., Bhowmik, P., Rangappa, S. M., & Siengchin, S. (2025) · Discover Applied Sciences · doi:10.1007/s42452-025-07616-8 · Cited by 100 · see citing works

Greed Is Good: Rapid Hyperparameter Optimization and Model Selection Using Greedy k-Fold Cross Validation

Soper, D. S. (2021) · Electronics · doi:10.3390/electronics10161973 · Cited by 87 · see citing works

Construction of a quality model for machine learning systems

Siebert, J., Jöckel, L., Heidrich, J., Trendowicz, A., Nakamichi, K., Ohashi, K., Namba, I., Yamamoto, R., & Aoyama, M. (2021) · Software Quality Journal · doi:10.1007/s11219-021-09557-y · Cited by 84 · see citing works

A Joint Study of the Challenges, Opportunities, and Roadmap of MLOps and AIOps: A Systematic Survey

Díaz-de-Arcaya, J., Torre-Bastida, A. I., Zárate, G., Miñón, R., & Almeida, A. (2023) · ACM Computing Surveys · doi:10.1145/3625289 · Cited by 105 · see citing works

Supervised Machine Learning Approaches for Predicting Key Pollutants and for the Sustainable Enhancement of Urban Air Quality: A Systematic Review

Essamlali, I., Nhaila, H., & Khaïli, M. E. (2024) · Sustainability · doi:10.3390/su16030976 · Cited by 81 · see citing works

A Review on AI for Smart Manufacturing: Deep Learning Challenges and Solutions

Xu, J., Kovatsch, M., Mattern, D., Mazza, F., Harasic, M., Paschke, A., & Lucia, S. (2022) · Applied Sciences · doi:10.3390/app12168239 · Cited by 86 · see citing works

Cognitive Architectures for Language Agents

Sumers, T. R., Yao, S., Narasimhan, K., & Griffiths, T. L. (2023) · arXiv · doi:10.48550/arxiv.2309.02427 · Cited by 72 · see citing works

A data-driven approach with explainable artificial intelligence for customer churn prediction in the telecommunications industry

Asif, D., Arif, M. S., & Mukheimer, A. A. (2025) · Results in Engineering · doi:10.1016/j.rineng.2025.104629 · Cited by 75 · see citing works

Software Engineering for AI-Based Systems: A Survey

Martínez‐Fernández, S., Bogner, J., Franch, X., Oriol, M., Siebert, J., Trendowicz, A., Vollmer, A. M., & Wagner, S. (2022) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3487043 · Cited by 374 · see citing works

Software Architecture Challenges for ML Systems

Lewis, G. A., Özkaya, İ., & Xu, X. (2021) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsme52107.2021.00071 · Cited by 57 · see citing works

A deep learning method for empirical spectral prediction and inverse design of all-optical nonlinear plasmonic ring resonator switches

Adibnia, E., Mansouri-Birjandi, M. A., Ghadrdan, M., & Jafari, P. (2024) · Scientific Reports · doi:10.1038/s41598-024-56522-3 · Cited by 55 · see citing works

A Meta-Summary of Challenges in Building Products with ML Components – Collecting Experiences from 4758+ Practitioners

Nahar, N., Zhang, H., Lewis, G. A., Zhou, S., & Kästner, C. (2023) · IEEE/ACM International Conference on AI Engineering – Software Engineering for AI (CAIN) · doi:10.1109/cain58948.2023.00034 · Cited by 58 · see citing works

Navigating challenges and opportunities of machine learning in hydrogen catalysis and production processes: Beyond algorithm development

Salehmin, M. N. I., Kiong, T. S., Mohamed, H., Umar, D. A., Yu, K. L., Ong, H. C., Nomanbhay, S. M., & Lim, S. S. (2024) · Journal of Energy Chemistry · doi:10.1016/j.jechem.2024.07.045 · Cited by 54 · see citing works

DevOps for AI – Challenges in Development of AI-enabled Applications

Lwakatare, L. E., Crnković, I., & Bosch, J. (2020) · International Conference on Software, Telecommunications and Computer Networks (SoftCOM) · doi:10.23919/softcom50211.2020.9238323 · Cited by 55 · see citing works

Green AI: Enhancing Sustainability and Energy Efficiency in AI-Integrated Enterprise Systems

Dash, S. (2025) · IEEE Access · doi:10.1109/access.2025.3532838 · Cited by 52 · see citing works

Systematic literature review on software quality for AI-based software

Gezici, B., & Tarhan, A. (2022) · Empirical Software Engineering · doi:10.1007/s10664-021-10105-2 · Cited by 47 · see citing works

Security for Machine Learning-based Software Systems: A Survey of Threats, Practices, and Challenges

Chen, H., & Babar, M. A. (2023) · ACM Computing Surveys · doi:10.1145/3638531 · Cited by 50 · see citing works

Multilayered review of safety approaches for machine learning-based systems in the days of AI

Dey, S., & Lee, S.-W. (2021) · Journal of Systems and Software · doi:10.1016/j.jss.2021.110941 · Cited by 37 · see citing works

Operationalizing machine learning models: a systematic literature review

Kolltveit, A. B., & Li, J. (2022) · Proceedings of the 1st Workshop on Software Engineering for Responsible AI · doi:10.1145/3526073.3527584 · Cited by 34 · see citing works

Artificial intelligence for collective intelligence: a national-scale research strategy

Bullock, S., Ajmeri, N., Batty, M., Black, M. M., Cartlidge, J., Challen, R., Chen, C., Jing, C., Condell, J. V., Danon, L., et al. (2024) · The Knowledge Engineering Review · doi:10.1017/s0269888924000110 · Cited by 33 · see citing works

A reference architecture for the operationalization of machine learning models in manufacturing

Raffin, T., Reichenstein, T., Werner, J., Kühl, A., & Franke, J. E. (2022) · Procedia CIRP · doi:10.1016/j.procir.2022.10.062 · Cited by 32 · see citing works

Model driven engineering for machine learning components: A systematic literature review

Naveed, H., Arora, C., Khalajzadeh, H., Grundy, J., & Haggag, O. (2024) · Information and Software Technology · doi:10.1016/j.infsof.2024.107423 · Cited by 38 · see citing works

From a Data Science Driven Process to a Continuous Delivery Process for Machine Learning Systems

Lwakatare, L. E., Crnkovic, I., Rånge, E., & Bosch, J. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-64148-1_12 · Cited by 28 · see citing works

Machine learning-enabled healthcare information systems in view of Industrial Information Integration Engineering

Uysal, M. P. (2022) · Journal of Industrial Information Integration · doi:10.1016/j.jii.2022.100382 · Cited by 28 · see citing works

Machine Learning With Computer Networks: Techniques, Datasets, and Models

Afifi, H., Pochaba, S., Boltres, A., Laniewski, D., Haberer, J., Paeleke, L., Poorzare, R., Stolpmann, D., Wehner, N., Redder, A., et al. (2024) · IEEE Access · doi:10.1109/access.2024.3384460 · Cited by 32 · see citing works

Gated recurrent unit (GRU)-based deep learning method for spectrum estimation and inverse modeling in plasmonic devices

Mahadi, M. K., Rahad, R., Haque, M. A., & Nishat, M. M. (2024) · Applied Physics A · doi:10.1007/s00339-024-07956-z · Cited by 25 · see citing works

Machine Learning in Cultivated Meat: Enhancing Sustainability, Efficiency, Quality, and Scalability Across the Production Pipeline

Ng, W. L., & Tan, J. S. (2025) · Food and Bioprocess Technology · doi:10.1007/s11947-025-03853-2 · Cited by 25 · see citing works

Which design decisions in AI-enabled mobile applications contribute to greener AI?

Castanyer, R. C., Martínez‐Fernández, S., & Franch, X. (2023) · Empirical Software Engineering · doi:10.1007/s10664-023-10407-7 · Cited by 24 · see citing works

A Systematic Mapping of Quality Models for AI Systems, Software and Components

Ali, M. A., Yap, N. K., Ghani, A., Zulzalil, H. B., Admodisastro, N. I., & Najafabadi, A. A. (2022) · Applied Sciences · doi:10.3390/app12178700 · Cited by 24 · see citing works

Strategies for predictive power: Machine learning models in city-scale load forecasting

Nooruldeen, O., Baker, M. R., Aleesa, A. M., Ghareeb, A. H., & Shaker, E. H. (2023) · e-Prime - Advances in Electrical Engineering Electronics and Energy · doi:10.1016/j.prime.2023.100392 · Cited by 22 · see citing works

Machine learning to assess and support safe drinking water supply: a systematic review

Feng, F., Zhang, Y., Chen, Z., Ni, J., Feng, Y., Xie, Y., & Zhang, C. (2024) · Journal of environmental chemical engineering · doi:10.1016/j.jece.2024.114481 · Cited by 22 · see citing works

One-Class Anomaly Detection for Industrial Applications: A Comparative Survey and Experimental Study

Paolini, D., Dini, P., Soldaini, E., & Saponara, S. (2025) · Computers · doi:10.3390/computers14070281 · Cited by 21 · see citing works

Enhancing friction stir-based techniques with machine learning: a comprehensive review

El-Zathry, N. E., Akinlabi, S. A., Woo, W. L., Patel, V. V., Mahamood, R. M., & Sabry, I. (2025) · Machine Learning Science and Technology · doi:10.1088/2632-2153/adcff6 · Cited by 21 · see citing works

A Comprehensive Review on Scaling Machine Learning Workflows Using Cloud Technologies and DevOps

Ramesh, G., Pai, T. V., Birau, R., Poojary, K. K., Abhay, Shingad, A. R., Sowjanya, N., Popescu, V., Mitroi, A. T., Chireac, R. M. N., et al. (2025) · IEEE Access · doi:10.1109/access.2025.3599281 · Cited by 21 · see citing works

Challenges in developing and deploying AI in the engineering, procurement and construction industry

Джусупова, Р., Bosch, J., & Olsson, H. H. (2022) · IEEE Annual International Computer Software and Applications Conference (COMPSAC) · doi:10.1109/compsac54236.2022.00167 · Cited by 22 · see citing works

Advancing mix design prediction in 3D printed concrete: Predicting anisotropic compressive strength and slump flow

Malik, U. J., Riaz, R. D., Rehman, S. U., Usman, M., Riaz, R. E., & Hamza, R. (2024) · Case Studies in Construction Materials · doi:10.1016/j.cscm.2024.e03510 · Cited by 25 · see citing works

A Survey on Data Quality Dimensions and Tools for Machine Learning Invited Paper

Zhou, Y., Tu, F., Sha, K., Ding, J., & Chen, H. (2024) · IEEE International Conference on Artificial Intelligence Testing (AITest) · doi:10.1109/aitest62860.2024.00023 · Cited by 21 · see citing works

Maintainability Challenges in ML: A Systematic Literature Review

Shivashankar, K., & Martini, A. (2022) · Euromicro Conference on Software Engineering and Advanced Applications (SEAA) · doi:10.1109/seaa56994.2022.00018 · Cited by 22 · see citing works

Intelligence frameworks for environmental pollution assessment: a review on air and water quality monitoring systems

Pati, A. K., Tripathy, A. R., & Subudhi, S. (2025) · Measurement · doi:10.1016/j.measurement.2025.119122 · Cited by 22 · see citing works

A Nonlinear Dimensionality Reduction Search Improved Differential Evolution for large-scale optimization

Yang, Y., Li, H., Lei, Z., Yang, H., & Wang, J. (2024) · Swarm and Evolutionary Computation · doi:10.1016/j.swevo.2024.101832 · Cited by 18 · see citing works

Classification and challenges of non-functional requirements in ML-enabled systems: A systematic literature review

Martino, V. D., & Palomba, F. (2025) · Information and Software Technology · doi:10.1016/j.infsof.2025.107678 · Cited by 18 · see citing works

Knowledge-Driven Industrial Intelligent System: Concept, Reference Model, and Application Direction

Sun, P. Z. H., Bao, Y., Ming, X., & Zhou, T. (2022) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2022.3188295 · Cited by 17 · see citing works

A comprehensive systematic review of machine learning in the retail industry: classifications, limitations, opportunities, and challenges

Hassan, D. O., & Hassan, B. A. (2024) · Neural Computing and Applications · doi:10.1007/s00521-024-10869-w · Cited by 16 · see citing works

Unraveling the complexity of organophosphorus pesticides: Ecological risks, biochemical pathways and the promise of machine learning

Dong, Z., Zhu, Y., Che, R., Chen, T., Liang, J., Xia, M., & Wang, F. (2025) · The Science of The Total Environment · doi:10.1016/j.scitotenv.2025.179206 · Cited by 16 · see citing works

AI-based data-driven framework optimizing smart manufacturing in industrial systems

Basingab, M. S. (2025) · Journal of Industrial Information Integration · doi:10.1016/j.jii.2025.100996 · Cited by 16 · see citing works

Practices for Managing Machine Learning Products: A Multivocal Literature Review

Alves, I., Leite, L., Meirelles, P., Kon, F., & Rocha, C. (2023) · IEEE Transactions on Engineering Management · doi:10.1109/tem.2023.3287759 · Cited by 18 · see citing works

Status Quo and Problems of Requirements Engineering for Machine Learning: Results from an International Survey

Alves, A. P. S., Kalinowski, M., Giray, G., Méndez, D., Lavesson, N., Azevedo, K., Villamizar, H., Escovedo, T., Lopes, H., Biffl, S., et al. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-49266-2_11 · Cited by 16 · see citing works

Performance Evaluation of Computer Vision Algorithms in a Programmable Logic Controller: An Industrial Case Study

Vieira, R., da Silva, D. B., Ribeiro, E., Perdigoto, L., & Coelho, P. J. (2024) · Sensors · doi:10.3390/s24030843 · Cited by 16 · see citing works

Developing and Operating Artificial Intelligence Models in Trustworthy Autonomous Systems

Martínez-Fernández, S., Franch, X., Jedlitschka, A., Oriol, M., & Trendowicz, A. (2021) · Lecture Notes in Business Information Processing · doi:10.1007/978-3-030-75018-3_14 · Cited by 22 · see citing works

Deep insight: an efficient hybrid model for oil well production forecasting using spatio-temporal convolutional networks and Kolmogorov–Arnold networks

Hu, Y., Xin, X., Yu, G., & Deng, W. (2025) · Scientific Reports · doi:10.1038/s41598-025-91412-2 · Cited by 15 · see citing works

Review on crack detection in civil infrastructure using structural health monitoring and machine learning techniques

Khan, M. Z., Shahzadi, M., Khan, A. A., Ali, U. A., Hassan, M. A. S., & Hussain, M. (2025) · Innovative Infrastructure Solutions · doi:10.1007/s41062-025-02147-y · Cited by 20 · see citing works

Optimizing Data Warehousing Performance through Machine Learning Algorithms in the Cloud

Ahmadi, S. (2023) · International Journal of Science and Research (IJSR) · doi:10.21275/sr231224074241 · Cited by 15 · see citing works

Resilience and Recovery Mechanisms for Software-Defined Networking (SDN) and Cloud Networks

Akinola, O. I., Olaniyi, O. O., Ogungbemi, O. S., Oladoyinbo, O. B., & Olisa, A. O. (2024) · Journal of Engineering Research and Reports · doi:10.9734/jerr/2024/v26i81234 · Cited by 15 · see citing works

LEP-QNN: Loan Eligibility Prediction Using Quantum Neural Networks

Innan, N., Marchisio, A., Bennai, M., & Shafique, M. (2025) · IEEE International Conference on Quantum Computing and Engineering (QCE) · doi:10.1109/qce65121.2025.00204 · Cited by 15 · see citing works

Causal inference in AI education: A primer

Forney, A., & Mueller, S. (2022) · Journal of Causal Inference · doi:10.1515/jci-2021-0048 · Cited by 14 · see citing works

Qualitative assessment of the impact of manufacturing-specific influences on Machine Learning Operations

Raffin, T., Reichenstein, T., Klier, D., Kühl, A., & Franke, J. E. (2022) · Procedia CIRP · doi:10.1016/j.procir.2022.10.063 · Cited by 14 · see citing works

A machine learning approach for predicting high risk hospitalized patients with COVID-19 SARS-Cov-2

Bottrighi, A., Pennisi, M. A., Roveta, A., Massarino, C., Cassinari, A., Betti, M., Bolgeo, T., Bertolotti, M., Rava, E., & Maconi, A. (2022) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-022-02076-1 · Cited by 15 · see citing works

Comparative analysis of deep learning algorithms for predicting construction project delays in Saudi Arabia

Alsulamy, S. H. (2025) · Applied Soft Computing · doi:10.1016/j.asoc.2025.112890 · Cited by 14 · see citing works

A Requirements Engineering Perspective to AI-Based Systems Development: A Vision Paper

Franch, X., Jedlitschka, A., & Martínez‐Fernández, S. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-29786-1_15 · Cited by 14 · see citing works

A Concise Account for Challenges of Machine Learning in Seakeeping

Majidiyan, H., Enshaei, H., & Howe, D. (2025) · Procedia Computer Science · doi:10.1016/j.procs.2025.02.009 · Cited by 13 · see citing works

MLOps in freight rail operations

Pineda-Jaramillo, J., & Viti, F. (2023) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2023.106222 · Cited by 13 · see citing works

WeldVGG: A VGG-Inspired Deep Learning Model for Weld Defect Classification from Radiographic Images with Visual Interpretability

López, G., Duque, P., Vega, E., Pizarro, F., Toro, J., & Parra, C. (2025) · Sensors · doi:10.3390/s25196183 · Cited by 12 · see citing works

Integration of Convolutional Neural Networks in Mobile Applications

Castanyer, R. C., Martínez‐Fernández, S., & Franch, X. (2021) · IEEE/ACM International Conference on AI Engineering – Software Engineering for AI (CAIN) · doi:10.1109/wain52551.2021.00010 · Cited by 13 · see citing works

HybridRepair: towards annotation-efficient repair for deep learning models

Li, Y., Chen, M., & Xu, Q. (2022) · ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA) · doi:10.1145/3533767.3534408 · Cited by 12 · see citing works

System Integrity Protection Schemes: Naming Conventions and the Need for Standardization

Stanković, S., Hillberg, E., & Ackeby, S. (2022) · Energies · doi:10.3390/en15113920 · Cited by 11 · see citing works

Machine Learning Approaches to 3D Models for Drug Screening

da Silva, V. A., Sharma, R., Shteinberg, E., Patel, V., Bhardwaj, L., Garay, T., Yu, B., & Willerth, S. M. (2023) · Biomedical Materials & Devices · doi:10.1007/s44174-023-00142-4 · Cited by 11 · see citing works

Optimizing beyond boundaries: empowering the salp swarm algorithm for global optimization and defective software module classification

Kassaymeh, S., Al‐Betar, M. A., Rjoubd, G., Fraihat, S., Abdullah, S., & Almasri, A. (2024) · Neural Computing and Applications · doi:10.1007/s00521-024-10131-3 · Cited by 11 · see citing works

Architectural tactics to achieve quality attributes of machine-learning-enabled systems: a systematic literature review

Indykov, V., Strüber, D., & Wohlrab, R. (2025) · Journal of Systems and Software · doi:10.1016/j.jss.2025.112373 · Cited by 11 · see citing works

Machine Learning and Data Science Project Management From an Agile Perspective

Uysal, M. P. (2021) · Advances in logistics, operations, and management science book series · doi:10.4018/978-1-7998-7872-8.ch005 · Cited by 11 · see citing works

Predictive Maintenance in Industrial Systems Using Machine Learning

Kumar, R., Mishra, M., Suman, S., & Bali, P. S. (2024) · International Journal of Innovative Science and Research Technology (IJISRT) · doi:10.38124/ijisrt/ijisrt24mar1367 · Cited by 11 · see citing works

AI Deployment Architecture: Multi-Case Study for Key Factor Identification

John, M. M., Olsson, H. H., & Bosch, J. (2020) · Asia-Pacific Software Engineering Conference (APSEC) · doi:10.1109/apsec51365.2020.00048 · Cited by 11 · see citing works

The Impact of Resource Allocation on the Machine Learning Lifecycle

Duda, S., Hofmann, P., Urbach, N., Völter, F., & Zwickel, A. (2023) · Business & Information Systems Engineering · doi:10.1007/s12599-023-00842-7 · Cited by 12 · see citing works

Human–Robot Collaborative Manufacturing Cell with Learning-Based Interaction Abilities

Baptista, J., Castro, A., Gomes, M., Amaral, P., Santos, V. M. F., da Silva, F. M. T. P., & de Oliveira, M. A. R. (2024) · Robotics · doi:10.3390/robotics13070107 · Cited by 10 · see citing works

Detection of intra-beat waves on ambulatory ECG using manifolds: An explainable deep learning approach

Plaza–Seco, C., Barner, K. E., Holgado–Cuadrado, R., Melgarejo-Meseguer, F.-M., Rojo‐Álvarez, J. L., & Blanco–Velasco, M. (2025) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2025.107523 · Cited by 10 · see citing works

The Role of Mobile Communications for Industrial Automation: Architecture, Applications and Challenges

Zeydan, E., Arslan, Ş. S., Türk, Y., Hewa, T. M., & Liyanage, M. (2025) · IEEE Open Journal of the Communications Society · doi:10.1109/ojcoms.2025.3595403 · Cited by 11 · see citing works

Untitled work

(2025) · doi:10.1016/j.iot.2026.101938 · Cited by 31 · see citing works

A comparative analysis of ML techniques for bug report classification

Laiq, M., bin Ali, N., Börstler, J., & Engström, E. (2025) · Journal of Systems and Software · doi:10.1016/j.jss.2025.112457 · Cited by 9 · see citing works

Bridging academia and industry: A comprehensive review of advances, gaps, and future directions of fault detection and diagnosis ( FDD ) systems in the chemical industry

Roy, S., Pal, P. K., Saha, S., Behera, N., & Lahiri, S. K. (2025) · The Canadian Journal of Chemical Engineering · doi:10.1002/cjce.25701 · Cited by 8 · see citing works

Tool wear detection using novel acoustic emission features and a two-stage Mann-Whitney U test

Nguyen, D.-T., & Kim, J.-M. (2025) · Applied Acoustics · doi:10.1016/j.apacoust.2025.110952 · Cited by 8 · see citing works

Eco-friendly synthesis of ZnO nanoparticles using Delonix elata extract with enhanced antibacterial activity

Nanthakumar, J., Palanisamy, Y., Palanisamy, S., Karuppusamy, M., Raja, M. R., Abbas, M. A. E., Alagarsamy, A., & Rahman, M. Z. (2025) · RSC Advances · doi:10.1039/d5ra05208d · Cited by 8 · see citing works

An End-to-End Framework for Productive Use of Machine Learning in Software Analytics and Business Intelligence Solutions

Figalist, I., Elsner, C., Bosch, J., & Olsson, H. H. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-64148-1_14 · Cited by 8 · see citing works

Machine Learning Systems: A Survey from a Data-Oriented Perspective

Cabrera, C., Paleyes, A., Thodoroff, P., & Lawrence, N. D. (2025) · ACM Computing Surveys · doi:10.1145/3769292 · Cited by 12 · see citing works

Optimizing machine learning operations in multi-cloud infrastructure: a framework for unified deployment management and topology discovery

Wei, H., Pañeda, X. G., & Salvachúa, J. (2025) · Cluster Computing · doi:10.1007/s10586-025-05584-7 · Cited by 10 · see citing works

Improving Time Series Regression Model Accuracy via Systematic Training Dataset Augmentation and Sampling

Ströbel, R., Mau, M., Puchta, A., & Fleischer, J. (2024) · Machine Learning and Knowledge Extraction · doi:10.3390/make6020049 · Cited by 7 · see citing works

Exploring ML testing in practice

Song, Q., Borg, M., Engström, E., Ardö, H., & Rico, S. (2022) · IEEE/ACM International Conference on AI Engineering – Software Engineering for AI (CAIN) · doi:10.1145/3522664.3528596 · Cited by 7 · see citing works

Unveiling Deeper Petrochemical Insights: Navigating Contextual Question Answering with the Power of Semantic Search and LLM Fine-Tuning

Saikia, K. P., Mukherjee, D., Mahapatra, S., Nandy, P., & Das, R. K. K. (2023) · International Conference on Computing, Communication, and Intelligent Systems (ICCCIS) · doi:10.1109/icccis60361.2023.10425564 · Cited by 7 · see citing works

Scalable earthquake magnitude prediction using spatio-temporal data and model versioning

Singh, R., & Roy, B. (2025) · Scientific Reports · doi:10.1038/s41598-025-00804-x · Cited by 6 · see citing works

Optimizing Cloud Environments: Machine Learning-Driven Virtual Machine Migration Strategies

Panesar, G. S., & Chadha, R. (2023) · International Conference on Technological Advancements in Computational Sciences (ICTACS) · doi:10.1109/ictacs59847.2023.10389990 · Cited by 6 · see citing works

Enhancing Software Requirements Classification with Semisupervised GAN‐BERT Technique

Airlangga, G. (2024) · Journal of Electrical and Computer Engineering · doi:10.1155/2024/4955691 · Cited by 6 · see citing works

State-of-the-Art Review of Taxonomies for Quality Assessment of Intelligent Software Systems

Kharlamova, A., Jabborov, A., Kruglov, A. V., & Succi, G. (2022) · International Informatics and Software Engineering Conference (IISEC) · doi:10.1109/iisec56263.2022.9998232 · Cited by 6 · see citing works

Positive feedback loops lead to concept drift in machine learning systems

Khritankov, A. (2023) · Applied Intelligence · doi:10.1007/s10489-023-04615-3 · Cited by 6 · see citing works

An empirical evaluation of flow based programming in the machine learning deployment context

Paleyes, A., Cabrera, C., & Lawrence, N. D. (2022) · IEEE/ACM International Conference on AI Engineering – Software Engineering for AI (CAIN) · doi:10.1145/3522664.3528601 · Cited by 6 · see citing works

Fixes That Fail: Self-Defeating Improvements in Machine-Learning SystemsPreprint

Wu, R., Guo, C., Hannun, A., & van der Maaten, L. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.11766 · Cited by 6 · see citing works

Scalability and Maintainability Challenges and Solutions in Machine Learning: A Systematic Literature Review

Shivashankar, K., Hajj, G. S. A., & Martini, A. (2023) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.8024833 · Cited by 2 · see citing works

Scalability and Maintainability Challenges and Solutions in Machine Learning: Systematic Literature Review

Shivashankar, K., Hajj, G. S. A., & Martini, A. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2504.11079

Maintainability and Scalability in Machine Learning: Challenges and Solutions

Shivashankar, K., Hajj, G. S. A., & Martini, A. (2025) · ACM Computing Surveys · doi:10.1145/3736751 · Cited by 35 · see citing works

Applied Machine Learning: Scalable Models for Key Industries

Kar, A., Bikshapathi, M. S., MATYAKUBOV, N., Akhmedov, S., & Sabirov, S. (2025) · International Conference on Sustainability, Innovation & Technology (ICSIT) · doi:10.1109/icsit65336.2025.11294507

Toward the Sustainable Development of Machine Learning Applications in Industry 4.0Applications in Industry 4.

Ellenrieder, S., Jourdan, N., Biegel, T., Cassoli, B. B., & Metternich, J. (2023) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.8151777