Challenges in Deploying Machine Learning: A Survey of Case Studies
Open accessThe DOI matches its Crossref record; 127 of the 128 references it cites were found in a registry or an index.
Abstract
In recent years, machine learning has transitioned from a field of academic research interest to a field capable of solving real-world business problems. However, the deployment of machine learning models in production systems can present a number of issues and concerns. This survey reviews published reports of deploying machine learning solutions in a variety of use cases, industries, and applications and extracts practical considerations corresponding to stages of the machine learning deployment workflow. By mapping found challenges to the steps of the machine learning deployment workflow, we show that practitioners face issues at each stage of the deployment process. The goal of this article is to lay out a research agenda to explore approaches addressing these challenges.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Compared with the first step, continual monitoring is used to identify data distribution drift, control model quality, and trigger system alarms when an ML model deviates from its normal behavior in the target setting [15].”
From Toward real‐world deployment of machine learning for health care: External validation, continual monitoring, and randomized clinical trials (2024) · cites this paper as [15]
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Our literature review was conducted using a systematic mapping protocol [54] in order to select a small set of relevant articles from the much larger collection of literature emerging at the intersection of data quality and ML.”
From A Survey of Data Quality Requirements That Matter in ML Development Pipelines (2023) · cites this paper as [54]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325 · cited 2×
“They took one of the state-of-the-art models in the field, BERT [47], and found out that depending on the chosen model size full training procedure can cost anywhere between $50k and $1.”
BackgroundQuinlan, J. R. (1986) · Machine Learning · doi:10.1007/bf00116251 · Cited by 21,439 · cited 1×
“For that reason decision trees (DT) [108], which can be considered a fairly basic ML algorithm, are widely used in practice.”
MethodArmstrong, J. S., & Georgoff, D. M. (1988) · International Journal of Forecasting · doi:10.1016/0169-2070(88)90019-2 · Cited by 17,894
Rudin, C. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0048-x · Cited by 10,561 · cited 1×
“Rudin [111] called the ML community to stop using black box models and explaining their behavior afterward, and instead design models that are inherently interpretable.”
BackgroundOn the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445922 · Cited by 6,988
Fully homomorphic encryption using ideal lattices
Gentry, C. (2009) · Proceedings of the forty-first annual ACM symposium on Theory of computing · doi:10.1145/1536414.1536440 · Cited by 6,806 · cited 1×
“Some of the ways this trade-of is resolved in practice are diferential privacy that explicitly corrupts the data [42], homomorphic encryption that restricts the class of learning algorithm but allows for training on encrypted data [50], and federated learning that distributes training across personal devices to preser…”
BackgroundPractical Bayesian Optimization of Machine Learning Algorithms
Snoek, J., Larochelle, H., & Adams, R. P. (2012) · arXiv · doi:10.48550/arxiv.1206.2944 · Cited by 6,691 · cited 1×
“Even approaches like Hyperband [78] or Bayesian optimization [129], that are speciically designed to minimize the number of training cycles needed, are not yet able to deal with the high dimensional searches that emerge when many hyper-parameters are involved.”
Dwork, C. (2006) · Lecture Notes in Computer Science · doi:10.1007/11787006_1 · Cited by 5,291 · cited 1×
“Some of the ways this tradeoff is resolved in practice are differential privacy that explicitly corrupts the data [54], homomorphic encryption that restricts the class of learning algorithm but allows for training on encrypted data [55], and federated learning that distributes training across personal devices to prese…”
BackgroundMembership Inference Attacks Against Machine Learning Models
Shokri, R., Stronati, M., Song, C., & Shmatikov, V. (2017) · 2017 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp.2017.41 · Cited by 4,754 · cited 1×
“They veriied it on models trained on leading ML-as-a-service providers [127], achieving anywhere from 70% up to 94% accuracy of membership inference.”
BackgroundA survey on concept drift adaptation
Gama, J., Žliobaitė, I., Bifet, A., Pechenizkiy, M., & Bouchachia, A. (2014) · ACM Computing Surveys · doi:10.1145/2523813 · Cited by 3,735 · cited 1×
“We refer interested readers to the works on using domain adaptation [48], meta-learning [46] and transfer learning [152, 157] as ways of addressing dataset shift.”
Method
Research works printed but not linked: 26
- Arif Cam, Michael Chui, and Bryce Hall. 2019. Global AI survey: AI proves its worth, but few scale impact. McKinsey Analytics (2019). https://www.mckinsey.com/featured-insights/artificial-intelligence/global-ai-survey-ai-proves-its-worth-but-few-scale-impact. Accessed from 13/06/2022.
- Royal Society (Great Britain). 2017. Machine Learning: The Power and Promise of Computers that Learn by Example: an Introduction. Royal Society.
- Ben Lorica and Nathan Paco. 2018. The State of Machine Learning Adoption in the Enterprise. O’Reilly Media.
- Colin Shearer. 2000. The CRISP-DM model: The new blueprint for data mining. J. Data Warehous. 5, 4 (2000), 13–22.
- Rajashree Y. Patil and R. V. Kulkarni. 2012. A review of data cleaning algorithms for data warehouse systems. Int. J. Comput. Sci. Inf. Technol. 3, 5 (2012), 5212–5214.
- 10.1109/TITS.2019.2962338 · DOI · Paper page
- Diego Charrez. 2019. NeurIPS 2019 Stats. Retrieved from https://medium.com/@dcharrezt/neurips-2019-stats-c91346d31c8f.
- 10.1023/A:1022643204877 · doi:10.1023/a:1022643204877 · Paper page
- 10.5555/3122009.3242042 · doi:10.5555/3122009.3242042
- Bobak Shahriari, Alexandre Bouchard-Cote, and Nando Freitas. 2016. Unbounded Bayesian optimization via regularization. In Proceedings of the 19th International Conference on Artificial Intelligence and Statistics (Proceedings of Machine Learning Research), Arthur Gretton and Christian C. Robert (Eds.), Vol. 51. PMLR, 1168–1176. Retrieved from http://proceedings.mlr.press/v51/shahriari16.html.
- Ram Ananth, Seph Mard, and Peter Simon. 2019. Opening the “Black Box”: The path to deployment of AI models in banking, white paper. DataRobot (2019). https://www.datarobot.com/resources/opening-the-black-box/. Accessed from 13/06/2022.
- Robert Soden Dennis Wagenaar Dave Luo and Annegien Tijssen. 2019. Taking Ethics Fairness and Bias Seriously in Machine Learning for Disaster Risk Management. arxiv:cs.CY/1912.05538.
- Nantheera Anantrasirichai and David Bull. 2021. Artificial intelligence in the creative industries: A review. Artif. Intell. Rev. (2021), 1–68.
- D. Malan et al. 2016. Values and the fourth industrial revolution: Connecting the dots between value, values, profit and purpose. In Proceedings of the World Economic Forum.
- Daniel Malan. 2018. The law can’t keep up with new tech. Here’s how to close the gap. In Proceedings of the World Economic Forum.
- Ramprakash Ramamoorthy P. Satya Madhuri and Malini Christina Raj. 2019. AI from labs to production—challenges and learnings. USENIX Association. https://www.usenix.org/conference/opml19/presentation/madhuri. Accessed from 13/06/2022.
- Oscar Schwartz. 2019. In 2016 Microsoft’s racist chatbot revealed the dangers of online conversation. IEEE Spectr. 11 (2019).
- Florian Tramèr, Fan Zhang, Ari Juels, Michael K. Reiter, and Thomas Ristenpart. 2016. Stealing machine learning models via prediction APIs. In Proceedings of the 25th USENIX Security Symposium (USENIX Security’16). 601–618.
- Jeremy Hermann and Mike Del Balso. 2017. Meet Michelangelo: Uber’s machine learning platform. Retrieved from https://eng.uber.com/michelangelo.
- Matei Zaharia, Andrew Chen, Aaron Davidson, Ali Ghodsi, Sue Ann Hong, Andy Konwinski, Siddharth Murching, Tomas Nykodym, Paul Ogilvie, Mani Parkhe, et al. 2018. Accelerating the machine learning lifecycle with MLflow.IEEE Data Eng. Bull. 41, 4 (2018), 39–45.
- Nick Hynes, D. Sculley, and Michael Terry. 2017. The Data Linter: Lightweight, automated sanity checking for ML data sets. In Proceedings of the NIPS MLSys Workshop.
- Matthias Feurer, Aaron Klein, Katharina Eggensperger, Jost Springenberg, Manuel Blum, and Frank Hutter. 2015. Efficient and robust automated machine learning. In Advances in Neural Information Processing Systems, C. Cortes, N. Lawrence, D. Lee, M. Sugiyama, and R. Garnett (Eds.), Vol. 28. Curran Associates, Inc. Retrieved from https://proceedings.neurips.cc/paper/2015/file/11d0e6287202fced83f79975ec59a3a6-Paper.pdf.
- Randal S. Olson and Jason H. Moore. 2016. TPOT: A tree-based pipeline optimization tool for automating machine learning. In Proceedings of the Workshop on Automatic Machine Learning. PMLR, 66–74.
- Stephan Rabanser, Stephan Günnemann, and Zachary Chase Lipton. 2019. Failing loudly: An empirical study of methods for detecting dataset shift. In Proceedings of the Neural Information Processing Systems Foundation Conference.
- Chelsea Finn, Pieter Abbeel, and Sergey Levine. 2017. Model-agnostic meta-learning for fast adaptation of deep networks. In Proceedings of the International Conference on Machine Learning. PMLR, 1126–1135.
- A. Zujus. 2018. AI project development–how project managers should prepare. Tratto il giorno Agosto 31 (2018), 2019.
Cited by
Items 401–500 of 737
Louw, T. R., & McIntosh‐Smith, S. (2022) · Communications in computer and information science · doi:10.1007/978-3-030-96498-6_3 · Cited by 2
Approximative Uncertainty in Neural Network Predictions
Malmström, M. (2023) · Linköping studies in science and technology. Dissertations · doi:10.3384/9789180754064 · Cited by 3
Machine Learning Experiment Management With MLFlow
Erden, C. (2023) · IGI Global eBooks · doi:10.4018/978-1-7998-9220-5.ch071 · Cited by 2
Marx, E., Benoit, W., Gunny, A., Omer, R., Chatterjee, D., Venterea, R. C., Wills, L., Saleem, M., Moreno, E. A., Raikman, R. J., et al. (2024) · Research Square · doi:10.21203/rs.3.rs-4271631/v1 · Cited by 2
Frenzel, J., Horn, C., Kerschensteiner, M., Knothe, K., Thekale, A., & Wassermann, D. (2023) · Iron & Steel Technology Conference and Exposition (AISTech) · doi:10.33313/387/255 · Cited by 1
A Real-Time Nut-Type Classifier Application Using Transfer Learning
Özçevik, Y. (2023) · Applied Sciences · doi:10.3390/app132111644 · Cited by 1
Using FactoryML for Deployment of Machine Learning Models in Industrial Production
Wewer, C. R., Mahapatra, H., Esterle, L., & Larsen, P. G. (2024) · IEEE International Conference on Emerging Technologies and Factory Automation (ETFA) · doi:10.1109/etfa61755.2024.10710938 · Cited by 1
“However, it is always challenging to move the completely trained models into the operational setting of the actual manufacturing processes [1].”
BackgroundEnhanced Log Anomaly Detection with Contrastive Learning and BERT Embeddings
Cui, M., Gao, T., Xiang, H., Xu, X., Lu, C., Xiong, J., & Xue, S. (2024) · IEEE International Symposium on Parallel and Distributed Processing with Applications (ISPA) · doi:10.1109/ispa63168.2024.00024 · Cited by 1
“Supervised and unsupervised models [17] have been explored, with sequence-to-sequence models showing promise in dynamically generating log templates.”
Mitigating Unfair Regression in Machine Learning Model Updates
Buselli, I., López, A. P., Jiménez, E. M., Anguita, D., Roli, F., & Oneto, L. (2024) · International Conference on Machine Learning and Applications · doi:10.1109/icmla61862.2024.00289 · Cited by 1
“…(ML) based tools in many different application areas such as finance [1], medicine [2], security [3], education [4], and justice [5], ML is requested to address more and more challenging technical problems [6] while societal awareness around the trustworthiness of these systems is rising [7].”
Parbat, S., Viedt, I., & Urbas, L. (2025) · Systems and Control Transactions · doi:10.69997/sct.122855 · Cited by 1
Improving the Transportability of a Deep Learning Denoising Model Using Transfer Learning Techniques
Quinones, L., & Tibi, R. (2025) · Seismological Research Letters · doi:10.1785/0220250053 · Cited by 1
Machine Learning Techniques in Expert Systems: Comparative Insights
Hassani, M. E., Faquih, L. E., & Machkour, N. (2025) · Procedia Computer Science · doi:10.1016/j.procs.2025.07.187 · Cited by 1
Verkerken, M., Callewaert, M., D’hooge, L., Wauters, T., Volckaert, B., & Turck, F. D. (2025) · IEEE European Symposium on Security and Privacy Workshops (EuroS&PW) · doi:10.1109/eurospw67616.2025.00030 · Cited by 2
“However, the adoption of network flow analysis for ML-based security applications remains challenging due to skepticism among security practitioners regarding ML-based traffic analysis [7]–[10].”
Afkhami, M. M., Moghadam, I. H., Zaytsev, V., Ashoori, M. H., & Bazmandegan, H. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-04190-6_4 · Cited by 1
Asymmetric impacts of artificial intelligence on housing price valuation across education levels
An, S., Song, Y., Jang, H., & Ahn, K. (2025) · Humanities and Social Sciences Communications · doi:10.1057/s41599-025-06153-4 · Cited by 1
“In other words, the deployed models require continuous monitoring and updates to be balanced in real-world settings, even after fi nalizing them during the testing phase (Gama et al., 2014; Paleyes et al., 2022).”
Next-Generation Biosensing Devices and the Use of AI/ML
Sharifi, F., Pérez, S. C. L., Erdi, A. K., & Çeçen, V. (2026) · Springer eBooks · doi:10.1007/978-981-95-5519-2_13 · Cited by 1
A Big Data-Enabled Model for Optimizing Corporate Governance Decisions and HR Allocation
Li, K., & Zheng, B. (2026) · Journal of Cases on Information Technology · doi:10.4018/jcit.405426 · Cited by 1
ZT-MCP: A Zero-Trust Security Architecture for MCP-Connected AI Agents
Mishra, S. (2026) · IEEE International Conference on Omni-layer Intelligent Systems (COINS) · doi:10.1109/coins69995.2026.11715596 · Cited by 2
“Deploying machine learning models in production consistently reveals 979-8-3195-0489-0/26/$31.00 © 2026 IEEE hidden technical debt [8], [9], and currently, no comprehensive framework addresses how agents securely discover, authenticate, and authorize external tools natively at the MCP protocol…”
InfluentialMalçok, H., BOZKIR, R., & Cicioğlu, M. (2026) · Future Generation Computer Systems · doi:10.1016/j.future.2026.108598 · Cited by 1
Gong, Y., Freitas, E. N. D. A., & Cabrera, V. E. (2026) · Sustainability · doi:10.3390/su18136900 · Cited by 1
“This emphasis is consistent with broader ML deployment literature: Paleyes et al. [68] survey the case-study literature on deploying machine learning at scale; Bayram et al. [71] formalize a taxonomy of performance-aware drift detectors; Sahiner et al. [69] articulate the implications of data drift…”
Data-Driven Suitability Analysis to Enable Machine Learning Explainability and Security
Wolf, S., Foster, R., Haile, J., & Borowczak, M. (2021) · Resilience Week (RWS) · doi:10.1109/rws52686.2021.9611792 · Cited by 1
QuickAutoML: Uma ferramenta para treinamento automatizado de modelos de aprendizado de máquina
Siqueira, G., Rodrigues, G., Feitosa, E. L., & Kreutz, D. (2021) · Escola Regional de Redes de Computadores (ERRC) · doi:10.5753/errc.2021.18547 · Cited by 1
Innovations Focusing on Artificial Intelligence in Computer Science Education
Erdağı, E. (2024) · Advances in educational technologies and instructional design book series · doi:10.4018/979-8-3693-4542-9.ch005 · Cited by 2
From Traditional to AI-Driven Exams: The Evolution of Online Education Assessment
Saha, S. (2025) · International Journal for Research in Applied Science and Engineering Technology · doi:10.22214/ijraset.2025.66795 · Cited by 1
The Role of AI in Data Anonymization: Can We Trust It with Our Privacy?
Bodimani, M. (2025) · International Journal For Multidisciplinary Research · doi:10.36948/ijfmr.2025.v07i02.42670 · Cited by 1
Shi, Z., Gong, D., & Yan, X. (2025) · Lecture notes in operations research · doi:10.1007/978-981-96-9697-0_77 · Cited by 2
Suwannoot, P., & Konghuayrob, P. (2025) · Sensors and Materials · doi:10.18494/sam5551 · Cited by 1
Q-SafeML: Safety Assessment of Quantum Machine Learning via Quantum Distance Metrics
Dunn, O., Aslansefat, K., & Papadopoulos, Y. I. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-05073-1_23 · Cited by 1
“One such method, SafeML , uses parametric distance measures to assess how closely a model’s predictions align with real-world data, helping detect weaknesses and uncertainty before or during deployment [24].”
InfluentialApplication of Machine Learning in Fault Detection and Predictive Maintenance of Power Transformers
Roy, J., Shahid, S., Alam, M. M., & Chanda, A. (2024) · Journal of Artificial Intelligence General science (JAIGS) · doi:10.60087/jaigs.v5i1.423 · Cited by 1
“Problems in the data availability, interpretability of the algorithms, and scalability have to be resolved to allow the deployment of the algorithms in safety-critical systems such as power transformers, as emphasized by Paleyes, Urma, and Lawrence (2023) [18].”
InfluentialToward Industrial Artificial Intelligence: A Discussion of Management, Technology, and Human
Zhang, W., & Bansal, S. (2026) · IEEE Systems Man and Cybernetics Magazine · doi:10.1109/msmc.2024.3520321 · Cited by 1
“IAI solutions in deployment are expected to be automatic and capable of dealing with various possible real-world scenarios [23], [100].”
Contextual Computing and AI Integration: Adaptive Decision Systems for Enterprise Environments
Rangasamy, T. (2026) · International Journal of Computational and Experimental Science and Engineering · doi:10.22399/ijcesen.4832 · Cited by 1
“Successful deployments require coordinating diverse stakeholder groups with different perspectives and priorities [10].”
InfluentialBhagavathula, M. B. (2025) · International Journal of AI BigData Computational and Management Studies · doi:10.63282/3050-9416.ijaibdcms-v6i2p112 · Cited by 1
A Feasibility Study of Literature-Guided HRV Stratification Using Large Language Models
Hsu, T. Y., Tang, G.-J., Wu, C.-H., Lee, J.-T., & Kuo, T. B. J. (2026) · Diagnostics · doi:10.3390/diagnostics16040540 · Cited by 1
“Because there is no way to know if there is an unknown bias in a model, opaque decision logics remain difficult to adopt [41,42].”
Liu, X., Zhao, J., Zhu, C., Yu, D. M., & Zhao, W. (2026) · Enterprise Information Systems · doi:10.1080/17517575.2026.2629561 · Cited by 1
Mishra, S. (2026) · IEEE International Conference on Omni-layer Intelligent Systems (COINS) · doi:10.1109/coins69995.2026.11715619 · Cited by 1
“Standard machine learning safety protocols and AI governance frameworks [3], [4] were designed for systems that make one prediction at a time, not for systems that act continuously across many steps.”
Perspectives On Machine Learning Inference Serving in Real‐World Settings
Muiruri, D., Lwakatare, L. E., Nurminen, J. K., & Mikkonen, T. (2026) · Software Practice and Experience · doi:10.1002/spe.70069 · Cited by 1
A systematic framework for preprocessing validation in business analytics
Garcia, M. J. J., & Herrera, R. D. J. G. (2026) · The Journal of Business Analysis · doi:10.1080/2573234x.2026.2634271
Jamal, S., Ahmad, A., & ZESHAN, D. F. (2026) · International Journal of Software Engineering and Knowledge Engineering · doi:10.1142/s0218194026300010 · Cited by 1
Perez, J. J., Peralta, J. M., Eugenio, A., Bueno, A. S., & Acerado, R. M. (2026) · International Conference on Software Engineering and Information Management (ICSIM) · doi:10.1145/3796315.3796339 · Cited by 1
Gough, H. (2026) · Innovative Journal of Applied Science · doi:10.70844/ijas.2026.3.50 · Cited by 1
DaFne: Data Fusion Generator and Synthetic Data Generation for Cities
Glass, A., Tokuç, K., Noennig, J. R., Steffens, U., & Bek, B. (2023) · Smart innovation, systems and technologies · doi:10.1007/978-981-99-3068-5_9 · Cited by 1
Mehare, H. B., Anilkumar, J. P., & Badar, M. S. (2023) · Springer eBooks · doi:10.1007/978-3-031-22206-1_8 · Cited by 1
Concept for the Evaluation and Prioritization of Machine Learning Use Cases in Industrial Production
Schuh, G., Cassel, L., & Uedelhoven, M. (2023) · IEEE International Conference on Industrial Engineering and Engineering Management (IEEM) · doi:10.1109/ieem58616.2023.10406937 · Cited by 1
“However, the implementation of these use cases in industrial practice is extremely time-consuming and capital-intensive [3].”
BackgroundSemantic Similarity Functions and Their Applications
Liu, Y., Alsaig, A., & Alagar, V. S. (2024) · Communications in computer and information science · doi:10.1007/978-981-97-0903-8_8 · Cited by 1
Overcoming the Challenges of Data Harmonization: A Platform Approach from Li-Ion Battery Research
Nebel, V., Mutz, M., Heim, Y., & Werth, D. (2024) · Lecture notes in networks and systems · doi:10.1007/978-981-99-8324-7_5 · Cited by 1
Approaching China’s “Artificial Intelligence Development Highland”
Todaro, D. (2024) · Springer eBooks · doi:10.1007/978-981-97-0597-9_1 · Cited by 1
Building and Evaluating a WebApp for Effortless Deep Learning Model Deployment
Wu, R., Han, J., Ramos, J., & Lipani, A. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-56069-9_26 · Cited by 1
Nykyta, K., Stanislav, M., Anton, G., & Tatjana, S. (2024) · International scientific and technical conference Information technologies in metallurgy and machine building · doi:10.34185/1991-7848.itmm.2024.01.080 · Cited by 2
Cruz, J. R. D., Miranda, V. R. R., & Naval, P. C. (2024) · IEEE International Conference on Cybernetics and Innovations (ICCI) · doi:10.1109/icci60780.2024.10532443 · Cited by 1
“However, while these results are promising, as comprehensively outlined in a recent survey of machine learning models [22], deep learning models face potential challenges in deployment especially due to computational cost and complexity [6].”
ResultExtreme Learning Machine Optimization for Employee Satisfaction With Modified Metaheuristic
Abdulla, T., Jovanović, L., Radomirović, B., Njeguš, A., Zivkovic, M. Z., & Bačanin, N. (2024) · International Conference on Expert Clouds and Applications (ICOECA) · doi:10.1109/icoeca62351.2024.00132 · Cited by 1
“Itisimperative to have high-quality training data in order to develop accurate models, while biases can serve as propagators or amplifiers of existing biases [6].”
BackgroundAspect Extraction in Domain Lexicon Generation: A New Frequency-Based Approach
Zayet, T. M. A., Ismail, M. A., & Varathan, K. D. (2024) · IEEE Access · doi:10.1109/access.2024.3442930 · Cited by 1
“While machine learning-based approaches give accurate results, it has multiple parameters to be set and in the case of the supervised approaches, labeled data is needed for training [15].”
BackgroundComputational Resource Optimisation in Feature Selection under Class Imbalance Conditions
Udu, A. G., Lecchini‐Visintini, A., Gunn, S. R., Osa-uwagboe, N., Ghalati, M. K., & Dong, H. (2024) · Proceedings of the Python in Science Conferences · doi:10.25080/tpgn6857 · Cited by 1
Research trends and hotspots in fundus image segmentation from 2007 to 2023: A bibliometric analysis
Deng, H., Wang, Y., He, Y., He, Y., Wen, Z., Chen, S., Guo, S., Zhou, P., & Wang, Y. (2024) · Heliyon · doi:10.1016/j.heliyon.2024.e39329 · Cited by 1
Interactive Ground-Truth-Free Image Selection for FLIM Segmentation Encoders
Cerqueira, M. A., Sprenger, F., Teixeira, B. C. D. A., Guimarães, S. J. F., & Falcão, A. X. (2024) · SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI) · doi:10.1109/sibgrapi62404.2024.10716300 · Cited by 1
“However, deep models usually rely on large annotated datasets and have become more complex, either by increasing the number of layers or by using different models together, demanding more powerful hardware, intensive human effort in data annotation, and energy consumption [1], [2].”
BackgroundMachine Learning-based Transformer Station Construction Safety Detection System
Zhang, D., Li, Q., He, J., Song, X., & Luo, Y. (2024) · International Conference on Power, Electrical Engineering, Electronics and Control (PEEEC) · doi:10.1109/peeec63877.2024.00039 · Cited by 1
Samonte, M. J. C., Chu, H. B., & Ogaya, L. A. T. (2024) · International Conference on Software Technology and Engineering · doi:10.1109/icste63875.2024.00040 · Cited by 1
“The overall effectiveness of the solution depends on the training and test data as much as on the algorithm [19].”
Deploying Predictive Models for a Process-Aware Decision Support System
Agarwal, P., & Sindhgatta, R. (2025) · Springer eBooks · doi:10.1007/978-3-031-80793-0_10 · Cited by 1
Flehmig, N., Lundteigen, M. A., & Yin, S. (2024) · Annual Conference of the IEEE Industrial Electronics Society (IECON) · doi:10.1109/iecon55916.2024.10906021 · Cited by 9
Sistem Keamanan Dua Lapis Dengan RFID dan Pendeteksi Objek Dengan Machine Learning
Jhonatan, J., & Sekarsari, K. (2024) · Building of Informatics Technology and Science (BITS) · doi:10.47065/bits.v6i2.5747 · Cited by 1
Make or buy strategy for Machine Learning Operations – MLOps
Nogare, D., Silveira, I. F., BANZAI, R., & ALEXANDRE, M. C. (2025) · Anais da Academia Brasileira de Ciências · doi:10.1590/0001-3765202520240924 · Cited by 1
A Modified Drake Equation for Assessing Adversarial Risk to Machine Learning Models
Kalin, J. D., Noever, D. A., Ciolino, M., & Dozier, G. V. (2021) · doi:10.5121/csit.2021.111001 · Cited by 1
Spectral Analysis of Neural Network Long-Term Predictions
Koparanov, K. A., Trifonov, R., Minkovska, D. V., & Georgiev, K. K. (2021) · International Conference on High Technology for Sustainable Development (HiTech) · doi:10.1109/hitech53072.2021.9614237 · Cited by 1
Crafting an Undergraduate Course at the Intersection of Machine Learning and Software Engineering
Chenoweth, S., & Linos, P. K. (2022) · IEEE Frontiers in Education Conference (FIE) · doi:10.1109/fie56618.2022.9962523 · Cited by 1
Analysis of Vulnerabilities of Neural Network Image Recognition Technologies
Trusov, A. V., Limonova, E. E., Arlazarov, V. V., & Zatsarinny, A. A. (2023) · Programming and Computer Software · doi:10.1134/s0361768823100079 · Cited by 1
Who is Responsible for Adversarial Defense?Preprint
Gupta, K. D., & Dasgupta, D. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.14152 · Cited by 1
“(Paleyes et al., 2020) provide a detailed case study of an ML model deployment challenge and also extend discussion on the adversarial challenge in ML deployment.”
BackgroundTest for non-negligible adverse shiftsPreprint
Kamulete, V. M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.02990 · Cited by 1
We Need to Talk About Data: The Importance of Data Readiness in Natural Language ProcessingPreprint
Olsson, F., & Sahlgren, M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.05464 · Cited by 2
“Research in this area that also touches on data readiness in some form include that of Polyzotis et al. (2018) and Paleyes et al. (2020).”
Rubo, R. A. (2022) · Digital Library of Theses and Dissertations (Universidade de São Paulo) · doi:10.11606/t.3.2022.tde-12082022-080122 · Cited by 1
Process Innovation with Analytics in Manufacturing: Exploration through Demonstrator Development
Bojer, C. S. (2024) · VBN Forskningsportal (Aalborg Universitet) · doi:10.54337/aau679672279 · Cited by 1
Nomogram for a machine learning model with categorical predictors to predict binary outcomePreprint
Sufriyana, H., & Su, E. C. (2024) · Research Square · doi:10.21203/rs.3.pex-2587/v1 · Cited by 1
Compression Method Matters: Benchmark-Dependent Output Dynamics in LLM Prompt Compression
Johnson, W. (2026) · arXiv · Cited by 1
“…that dynamically adjust compression ratios based on observed output-to-input token ratios, integrating with MLOps monitoring pipelines (Sculley et al., 2015; Paleyes et al., 2022) and A/B testing frameworks to continuously validate compression configurations against cost and quality objectives.”
Human-Clinical AI Agent Collaboration
Kadem, M., & Al-Khazraji, B. (2025) · Proceedings of the AAAI Symposium Series · doi:10.1609/aaaiss.v6i1.36060
“Such simulations would also provide testbeds for human-AI interaction theories (Card 2017) to improve decision making, trans-parency, accountability, reduce cognitive load and blind acceptance (Paleyes, Urma, and Lawrence 2022) of AI recommendations.”
Aiello, G. (2020) · Digital Commons - URI (University of Rhode Island) · doi:10.23860/thesis-aiello-gabriella-2020
Generatively Augmented Neural Network Watchdog for Image Classification Networks
Bui, J. M., Amigo, G., & Marks, R. J. (2023) · IEEE Annual Computing and Communication Workshop and Conference (CCWC) · doi:10.1109/ccwc57344.2023.10099329
Rodríguez-García, I., Sánchez-Pastor, T., Escobar, J. V., Gómez-González, J. L., & Cárdenas‐Montes, M. (2022) · Studies in computational intelligence · doi:10.1007/978-3-031-04597-4_1
Actor-oriented approach for business-process management of analytical system development
Pribylev, T., Zaytsev, M. S., & Vikentyeva, O. (2022) · Proceedings of the Institute for System Programming of RAS · doi:10.15514/ispras-2022-34(2)-5
“Meanwhile, an important part of the life cycle of ML models is monitoring and modification of models in case deviations are identified [10], which leads the model to return to the previous phases within the business processes.”
BackgroundMethodInfluentialYoung, A. A., Petersen, S. E., & Lamata, P. (2023) · Springer eBooks · doi:10.1007/978-3-031-05071-8_9
Web Based Portal for Complete Data Engineering
Dwivedi, P., Jadhav, S., Jadhav, S., & Singh, V. (2023) · International Journal for Research in Applied Science and Engineering Technology · doi:10.22214/ijraset.2023.48943
Adopting Machine-Learning-Based Workflows for Reducing Production Risk and Cost
Ozbay, P., & Almquist, C. (2023) · Iron & Steel Technology Conference and Exposition (AISTech) · doi:10.33313/387/235
Lin, P., Tongdecharoen, W., & Tasnaina, N. (2023) · International Journal of Sociologies and Anthropologies Science Reviews · doi:10.60027/ijsasr.2023.3729 · Cited by 1
Marvi, M., Aijaz, A., Qureshi, A., & Khurram, M. (2024) · Journal of Computer Networks and Communications · doi:10.1155/2024/9571987
“Moreover, according to the authors in [5], advanced AI techniques are not yet been considered for deployment in wireless networks because of the resource-constrained nature of these networks.”
BackgroundProcess Innovation with Analytics in Manufacturing: Exploration through Demonstrator Development
Bojer, C. S. (2024) · VBN Forskningsportal (Aalborg Universitet) · doi:10.54337/aau679682469
Data Related Challenges to Deploying ML Systems in Production: An Example from Sewer Inspection
Biswas, R., Nebel, V., & Werth, D. (2024) · Lecture notes in networks and systems · doi:10.1007/978-3-031-53960-2_15
Reinforcement Learning Circuit Based on Memristive Neural Network
Yue, Y., Chen, B., Zhai, Y., Sun, J., & Wang, Y. (2023) · IEEE International Conference on Memristive Computing and Applications (ICMCA) · doi:10.1109/icmca59770.2023.10481062
“While contemporary neural network algorithms have demonstrated remarkable problem-solving capabilities, most current AI technologies primarily excel in data classification tasks and lack genuine ideation and cognition [3], [4].”
BackgroundAboudi, N. E., Riouali, Y., Maminou, A. R., Jabri, H., & Benhlima, L. (2024) · International Journal of Power Electronics and Drive Systems/International Journal of Electrical and Computer Engineering · doi:10.11591/ijece.v14i3.pp3159-3167
“Deploying ML models into production with confidence and trust is a challenging task, requiring seamless integration with organizational processes and practices [22], [23].”
BackgroundApplication of Machine Learning Algorithm in Computer Data Remote Backup System
Chen, M. (2024) · International Conference on Integrated Circuits and Communication Systems (ICICACS) · doi:10.1109/icicacs60521.2024.10498295
“Traditional machine learning includes algorithms such as decision trees, random forests, artificial neural networks, and Bayesian learning [6-7].”
MethodComputer Science Data Structure System with Machine Learning Algorithms
Yu, X. (2024) · International Conference on Integrated Circuits and Communication Systems (ICICACS) · doi:10.1109/icicacs60521.2024.10499007
“Paleyes A proposed that machine learning has transformed from an academic research field to a field that can solve real-world business problems, and the ability to select data features is highly worth studying [10].”
BackgroundBest Practices for Secure Model Deployment on AWS
Bagai, R. (2024) · International Journal of Computer Science and Engineering · doi:10.14445/23488387/ijcse-v11i5p102
“Thus, having encryption and identity management mechanisms will enable the AWS platform to control the data revealed to the ML models, creating a suitable scope for addressing pending issues within the AWS platform [18].”
Scoping and Identifying Data-Driven Optimization Prospects in the Danish Processing Industry
Aouichaoui, A. R. N., Ernstsson, B. B., Jul‐Rasmussen, P., Iversen, N. H., Vermue, L., & Huusom, J. K. (2024) · Computer-aided chemical engineering/Computer aided chemical engineering · doi:10.1016/b978-0-443-28824-1.50076-4
Krasniqi, R. (2024) · University of North Texas Digital Library (University of North Texas) · doi:10.12794/metadc2332577
Navigating Data Engineering Obstacles in AI Automation
Verma, R., & Shrivastava, P. (2024) · International Journal of Global Innovations and Solutions (IJGIS) · doi:10.21428/e90189c8.ebeec245 · Cited by 1
Enhancing Collaboration and Agility in Data-Centric AI Projects
Stieler, F., & Bauer, B. (2024) · Communications in computer and information science · doi:10.1007/978-3-031-64182-4_15 · Cited by 1
A Pipeline for Monitoring and Maintaining a Text Classification Tool in Production
Ohata, E. F., Mattos, C. L. C., & Rêgo, P. A. L. (2024) · Anais do Seminário Integrado de Software e Hardware (SEMISH) · doi:10.5753/semish.2024.2438 · Cited by 1
“In that sense, besides asking how ML can transform a given procedure, recent ML literature has shed light on the question of how difficult is to deploy models in practice [Paleyes et al. 2022].”
ResultAdvancements in Deep Learning Architectures for Natural Language Processing Tasks
Reddy, E. K. (2024) · International Journal of Computer Science and Engineering · doi:10.14445/23488387/ijcse-v11i6p101 · Cited by 1
The application and benefits of emerging digital technologies for Industry 4.0
Govender, N. (2024) · DUT Open Scholar (Durban University of Technology) · doi:10.51415/10321/5455
A Designing Databases Framework for AI Training in Industrial Predictive Maintenance
Torres, G. M., Bessa, A. R. R., Linhares, J. E. B. D. S., Silva, M. O., Santos, A. V. D., Valadão, M. D. M., Menezes, V. S., Mattos, E. V. C. U., Pereira, A. M. C., Silva, A. P., et al. (2024) · IEEE International Conference on Consumer Electronics - Taiwan (ICCE-Taiwan) · doi:10.1109/icce-taiwan62264.2024.10674079
“techniques have been gradually applied to solve business, commercial, and industrial demands [1]–[3].”
BackgroundMethodMoshawrab, M., Adda, M., Bouzouane, A., Ibrahim, H., & Raad, A. (2024) · Electronics · doi:10.3390/electronics13183716 · Cited by 1
Study on Exam Paper Generation Methods and Experiments based on Machine Learning
Liu, M. (2024) · International Conference on Intelligent Education and Computer Technology (IECT) · doi:10.1145/3687311.3687398
de Souza, L. A. C., Sammarco, M., Achir, N., Campista, M. E. M., & Costa, L. H. M. K. (2024) · doi:10.5753/sbrc.2024.1514
“No entanto, a implantac¸ ˜ ao de modelos de intelig ˆ encia artificial em ambientes de produc¸ ˜ ao demora dias ou at ´ e meses [Paleyes et al. 2022].”
BackgroundZhang, N., Yongjia, L., Ji, P. W., & Josie, L. Z. X. (2024) · International Conference on Infrared, Millimeter, and Terahertz Waves (IRMMW-THz) · doi:10.1109/irmmw-thz60956.2024.10697925
Metadata
- Authors
- Paleyes, A., Urma, R.-G., & Lawrence, N. D.
- Affiliations
- University of Cambridge (GB)
- Cambridge Spark, Cambridge, United Kingdom
- Venue
- ACM Computing Surveys
- Published
- 2022-12-07
- Volume
- 55
- Issue
- 6
- Pages
- 1-29
- ISSN
- 0360-0300 (ISSN-L)
- 1557-7341
- OpenAlex
- W3104128335
- Semantic Scholar
- b1f574c47d0b6e3032246770b9cbebb9c7bd0c7f
- Cited by
- 638
- References
- 128
- Citations by year
- 2026: 149 · 2025: 156 · 2024: 166 · 2023: 93 · 2022: 46
- FWCI
- 114.17
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- ACM Computing Surveys (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- White Rose Research Online (University of Leeds, The University of Sheffield, University of York) (submittedVersion)
- Apollo (University of Cambridge) (submittedVersion)
- arXiv (Cornell University)
- Apollo
- Funders
- Alan Turing Institute
- UK Research & Innovation
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence Applications
- Physical Sciences › Computer Science › Information Systems › Cloud Computing and Resource Management
- Keywords
- machine learning deployment, production systems, deployment challenges, industry case studies
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.