Applying DevOps Practices of Continuous Automation for Machine Learning
Open accessThe DOI matches its Crossref record; 16 of the 16 references it cites were found in a registry or an index.
Abstract
This paper proposes DevOps practices for machine learning application, integrating both the development and operation environment seamlessly. The machine learning processes of development and deployment during the experimentation phase may seem easy. However, if not carefully designed, deploying and using such models may lead to a complex, time-consuming approaches which may require significant and costly efforts for maintenance, improvement, and monitoring. This paper presents how to apply continuous integration (CI) and continuous delivery (CD) principles, practices, and tools so as to minimize waste, support rapid feedback loops, explore the hidden technical debt, improve value delivery and maintenance, and improve operational functions for real-world machine learning applications.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures

![Figure 2 CRISP-DM methodology (Source: [26]).](https://figures.cochat.ai/paper-figures/a12adec26fd648cb876870d34b76a13d/applying-devops-practices-of-continuous-automation-for-machine-learning-figure-2.png)
![Figure 3 SEMMA methodology (Source: Adapted [27]). Figure 3. SEMMAmethodology (Source: Adapted [27]).](https://figures.cochat.ai/paper-figures/81d282ad0390445ba6f400b6e39f28d6/applying-devops-practices-of-continuous-automation-for-machine-learning-figure-3.png)
![Figure 4 TDSP methodology (Source [28]). Figure 4. TDSP methodology (Source [28]).](https://figures.cochat.ai/paper-figures/45b1c2a73786479e8034398452b468ee/applying-devops-practices-of-continuous-automation-for-machine-learning-figure-4.png)


Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Firstly, several related studies [18, 15, 16, F42, 6, 9, 21, 22, 26] are based on a limited amount of identified primary sources, not covering relevant insights from the wide range of existing literature.”
From The pipeline for the continuous development of artificial intelligence models—Current state of research and practice (2023) · cites this paper as [18]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Additionally, monitoring of the ML infrastructure, CI/CD, and orchestration are required[7],[23],[24],[28],[32],[33], [50], [α, ζ, η, θ].”
From Machine Learning Operations (MLOps): Overview, Definition, and Architecture (2023) · cites this paper as [24]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
(2015) · neural information processing systems · Cited by 912
Continuous software engineering: A roadmap and agenda
Fitzgerald, B., & Stol, K.-J. (2017) · Journal of Systems and Software · doi:10.1016/j.jss.2015.06.063 · Cited by 723 · cited 3×
“The automated CI/CD system allows the data scientists to rapidly explore new ideas around feature engineering, model architecture, and hyperparameters.”
BackgroundInfluentialEbert, C., Gallardo, G., Hernantes, J., & Serrano, N. (2016) · IEEE Software · doi:10.1109/ms.2016.68 · Cited by 504
Chacon, S., & Straub, B. (2014) · Apress eBooks · doi:10.1007/978-1-4842-0076-6 · Cited by 292
Continuous Delivery: Huge Benefits, but Challenges Too
Chen, L. (2015) · IEEE Software · doi:10.1109/ms.2015.27 · Cited by 261 · cited 1×
“According to [11,12], this practice offers several benefits such as reduced deployment risk, lower costs, and getting user feedback faster.”
BackgroundA qualitative study of DevOps usage in practice
Erich, F. M. A., Amrit, C., & Daneva, M. (2017) · Journal of Software: Evolution and Process · doi:10.1002/smr.1885 · Cited by 203 · cited 1×
“In this study, the TDSP methodology is selected.”
MethodThe highways and country roads to continuous deployment
Leppanen, M., Makinen, S., Pagels, M., Eloranta, V.-P., Itkonen, J., Mantyla, M. V., & Mannisto, T. (2015) · IEEE Software · doi:10.1109/ms.2015.50 · Cited by 150 · cited 1×
“This practice includes automated software building and testing [7].”
MethodUnderstanding DevOps & bridging the gap from continuous integration to continuous delivery
Virmani, M. (2015) · Fifth International Conference on the Innovative Computing Technology (INTECH 2015) · doi:10.1109/intech.2015.7173368 · Cited by 132 · cited 2×
“In addition, several researchers (Karamitsos [21], Virmani, [22]; Erich [23]; Lwakatare [24]) have agreed that Agile transformation is essential to improve the efficiency of the companies to optimize the lifecycle delivery, to break the gap and to create a continuous feedback loop between the business users and develo…”
BackgroundRelationship of DevOps to Agile, Lean and Continuous Deployment
Lwakatare, L. E., Kuvaja, P., & Oivo, M. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-49094-6_27 · Cited by 122 · cited 1×
“In this study, the TDSP methodology is selected.”
MethodContinuous Software Engineering
Bosch, J. (2014) · Springer eBooks · doi:10.1007/978-3-319-11283-1 · Cited by 109
Research works printed but not linked: 3
- 25. Forsgren, N.; Humble, J.; Gene, K. The Science of Lean Software and DevOps: Building and Scaling High Performing Technology Organizations ; IT Revolution Press: Portland, OR, USA, 2018.
- 26. Shearer, D. The CRISP-DM Model: The new Blueprint for Data Mining. J. Data Wareh. 2000 , 5 , 13-18.
- 29. Humble, J.; Farley, D. Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation ; Pearson Education, Inc.: Boston, MA, USA, 2011.
Cited by
Machine Learning Operations (MLOps): Overview, Definition, and Architecture
Kreuzberger, D., Kühl, N., & Hirschl, S. (2023) · IEEE Access · doi:10.1109/access.2023.3262138 · Cited by 766
“The solution architect designs the architecture and defines the technologies to be used, following a thorough evaluation [17,27] [α, ζ].”
BackgroundMethodInfluentialHow to explain AI systems to end users: a systematic literature review and research agenda
Laato, S., Tiainen, M., Najmul Islam, A. K. M., & Mäntymäki, M. (2022) · Internet Research · doi:10.1108/intr-08-2021-0600 · Cited by 155
MLOps - Definitions, Tools and Challenges
Symeonidis, G., Nerantzis, E., Kazakis, A., & Papakostas, G. A. (2022) · IEEE Annual Computing and Communication Workshop and Conference (CCWC) · doi:10.1109/ccwc54503.2022.9720902 · Cited by 227
“With this practice, the software releases resulting from the continuous integration with the improvements and the new features reach the end users much faster [17].”
MethodTowards MLOps: A Framework and Maturity Model
John, M. M., Olsson, H. H., & Bosch, J. (2021) · 2021 47th Euromicro Conference on Software Engineering and Advanced Applications (SEAA) · doi:10.1109/seaa53835.2021.00050 · Cited by 130
Steidl, M., Felderer, M., & Ramler, R. (2023) · Journal of Systems and Software · doi:10.1016/j.jss.2023.111615 · Cited by 126
“[18] base their CI/CD pipeline for AI on a literature review focusing on ”traditional” DevOps principles, not covering AI specific tasks due to the non-existence in DevOps pipelines.”
BackgroundMLOps: A Taxonomy and a Methodology
Testi, M., Ballabio, M., Frontoni, E., Iannello, G., Moccia, S., Soda, P., & Vessio, G. (2022) · IEEE Access · doi:10.1109/access.2022.3181730 · Cited by 114
“The goal of DevOps is to study ways to improve service quality and features in order to meet customer needs [23], [24].”
BackgroundDemystifying MLOps and Presenting a Recipe for the Selection of Open-Source Tools
Ruf, P., Madan, M., Reich, C., & Ould-Abdeslam, D. (2021) · Applied Sciences · doi:10.3390/app11198861 · Cited by 112
“The goal of CD is to get the new features developed to the end user as soon as possible [27,28].”
BackgroundGarg, S., Pundir, P., Rathee, G., Gupta, P. K., Garg, S., & Ahlawat, S. (2021) · 2021 IEEE Fourth International Conference on Artificial Intelligence and Knowledge Engineering (AIKE) · doi:10.1109/aike52691.2021.00010 · Cited by 91
DevOps critical success factors — A systematic literature review
Azad, N., & Hyrynsalmi, S. (2023) · Information and Software Technology · doi:10.1016/j.infsof.2023.107150 · Cited by 155
From DevOps to MLOps: Overview and Application to Electricity Market Forecasting
Subramanya, R., Sierla, S., & Vyatkin, V. V. (2022) · Applied Sciences · doi:10.3390/app12199851 · Cited by 109
“Karamitsos et al. [66] discussed such concepts and issues of applying DevOps practices to ML applications, while John et al. [62] discusses DevOps applications in ML systems.”
BackgroundMethodCONFIGURATION MANAGEMENT IN THE MODERN ERA: BEST PRACTICES, INNOVATIONS, AND CHALLENGES
Farayola, O. A., Hassan, A. O., Adaramodu, O. R., Fakeyede, O. G., & Oladeinde, M. (2023) · Computer Science & IT Research Journal · doi:10.51594/csitrj.v4i2.613 · Cited by 84
Donca, I.-C., Stan, O. P., Misaroş, M., Gota, D.-I., & Miclea, L. C. (2022) · Sensors · doi:10.3390/s22124637 · Cited by 102
“Practically, this step prepares, wraps, and pushes the application to go to the next step [30,31].”
BackgroundAbiona, O. O., Oladapo, O. J., Modupe, O. T., Oyeniran, O. C., Adewusi, A. O., & Komolafe, A. M. (2024) · World Journal of Advanced Engineering Technology and Sciences · doi:10.30574/wjaets.2024.11.2.0093 · Cited by 48
“The DevOps pipeline is a set of practices and tools used to automate the software delivery process, from code development to deployment and operations (Karamitsos et al ., 2020).”
MethodTowards Trustworthy Machine Learning in Production: An Overview of the Robustness in MLOps Approach
Bayram, F., & Ahmed, B. S. (2025) · ACM Computing Surveys · doi:10.1145/3708497 · Cited by 48
“EU’s requirement for trustworthy AI inspired by [38, 83].”
MLOps best practices, challenges and maturity models: A systematic literature review
Zarour, M., Alzabut, H., & Al-Sarayreh, K. T. (2025) · Information and Software Technology · doi:10.1016/j.infsof.2025.107733 · Cited by 91
Capabilities and Practices in DevOps: A Multivocal Literature Review
Amaro, R., Pereira, R., & da Silva, M. M. (2023) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2022.3166626 · Cited by 43
“They allow DevOps, developers, and system admin-istrators to swiftly, securely, and effectively test, build, deploy, and manage applications [69].”
BackgroundInfluentialUnlocking the Potential of AI/ML in DevSecOps: Effective Strategies and Optimal Practices
Camacho, N. G. (2024) · Journal of Artificial Intelligence General science (JAIGS) ISSN:3006-4023 · doi:10.60087/jaigs.v2i1.p89 · Cited by 42
Laato, S., Birkstedt, T., Mäntymäki, M., Minkkinen, M., & Mikkonen, T. (2022) · IEEE/ACM International Conference on AI Engineering – Software Engineering for AI (CAIN) · doi:10.1145/3522664.3528598 · Cited by 76
What drives MLOps adoption? An analysis using the TOE framework
Das, S. D., & Bala, P. K. (2023) · Journal of Decision System · doi:10.1080/12460125.2023.2214306 · Cited by 57
“Time-consuming and manual approaches might require more effort and cost for maintenance and monitoring (Karamitsos et al., 2020).”
The future of Artificial Intelligence for the BioTech Big Data landscape
Artico, F., Edge, A. L., & Langham, K. (2022) · Current Opinion in Biotechnology · doi:10.1016/j.copbio.2022.102714 · Cited by 24
Beyond the Buzz: A Journey Through CI/CD Principles and Best Practices
Thatikonda, V. K. (2023) · European Journal of Theoretical and Applied Sciences · doi:10.59324/ejtas.2023.1(5).24 · Cited by 19
An empirical study on ML DevOps adoption trends, efforts, and benefits analysis
Rzig, D. E., Hassan, F., & Kessentini, M. (2022) · Information and Software Technology · doi:10.1016/j.infsof.2022.107037 · Cited by 31
“Furthermore, they require new development processes and practices such as data engineering and model management [11, 26, 27], follow different collaboration strategies between their collaborators [20, 28], and may require different approaches to existing software development processes in comparison…”
The AIQ Meta-Testbed: Pragmatically Bridging Academic AI Testing and Industrial Q Needs
Borg, M. (2021) · Lecture notes in business information processing · doi:10.1007/978-3-030-65854-0_6 · Cited by 20
MLOps: a guide to its adoption in the context of responsible AI
Matsui, B. M. A., & Goya, D. (2022) · doi:10.1145/3526073.3527591 · Cited by 32
“Other studies such as [11], [12], and [27] can also be seen, which approach the concept of MLOps through tools and platforms that aim to automate the implementation of machine learning models.”
BackgroundMLOps Components, Tools, Process, and Metrics: A Systematic Literature Review
Woźniak, A. P., Milczarek, M., & Woźniak, J. (2025) · IEEE Access · doi:10.1109/access.2025.3534990 · Cited by 26
What Are Critical Success Factors of DevOps Projects? A Systematic Literature Review
Azad, N., & Hyrynsalmi, S. (2021) · Lecture notes in business information processing · doi:10.1007/978-3-030-91983-2_17 · Cited by 19
Soongpol, B., Netinant, P., & Rukhiran, M. (2024) · Sustainability · doi:10.3390/su16135738 · Cited by 37
Tran, T.-A., Ruppert, T., & Abonyi, J. (2024) · Computers · doi:10.3390/computers13100252 · Cited by 19
Díaz-de-Arcaya, J., Torre-Bastida, A. I., Miñón, R., & Almeida, A. (2022) · Future Generation Computer Systems · doi:10.1016/j.future.2022.10.008 · Cited by 25
Kumar, A., Nadeem, M., & Shameem, M. (2023) · Automated Software Engineering · doi:10.1007/s10515-023-00388-8 · Cited by 19
“DevOps teams can save a significant amount of time by using ML-driven requirements management solutions, allowing them to focus more on producing software products on time (Karamitsos et al. 2020).”
BackgroundCompliance and Audit Challenges in DevOps: A Security Perspective
(2023) · International Research Journal of Modernization in Engineering Technology and Science · doi:10.56726/irjmets45309 · Cited by 22
Comprehensive Analysis of DevOps: Integration, Automation, Collaboration, and Continuous Delivery
Moeez, M., Mahmood, R., Asif, H., Iqbal, M. W., Hamid, K., Ali, U., & Khan, N. H. (2024) · Bulletin of Business and Economics (BBE) · doi:10.61506/01.00253 · Cited by 21
“Currently, DevOps is adopted by three distinct groups, evident in the rapid increase in release frequency (Karamitsos et al., 2020).”
BackgroundMLOps approach in the cloud-native data pipeline design
Pölöskei, I. (2021) · Acta Technica Jaurinensis · doi:10.14513/actatechjaur.00581 · Cited by 13
“According to the MLOps principle, the Machine Learning tasks and artifacts can be combined with mainstream DevOps instruments [13].”
BackgroundMethodInfluentialFederated Learning Operations (FLOps): Challenges, Lifecycle and Approaches
Cheng, Q., & Long, G. (2022) · Conference on Technologies and Applications of Artificial Intelligence (TAAI) · doi:10.1109/taai57707.2022.00012 · Cited by 15
“Karamitsos et al. [13] applies DevOps practices of continuous automation for machine learning.”
BackgroundAutomated Machine Learning Deployment Using Open-Source CI/CD Tool
Parihar, A. S., Gupta, U. C., Srivastava, U., Yadav, V., & Trivedi, V. K. (2023) · Lecture notes in networks and systems · doi:10.1007/978-981-19-7615-5_19 · Cited by 16
FLScalize: Federated Learning Lifecycle Management Platform
Yang, S. H., Moon, J., Kim, J., Lee, K., & Lee, K. Y. (2023) · IEEE Access · doi:10.1109/access.2023.3275439 · Cited by 13
“MLOps aims to serve models accurately and reliably by creating a pipeline to deliver ML models [18].”
MethodModelOps for enhanced decision-making and governance in emergency control rooms
Lefevre, K., Arora, C., Lee, K., Zaslavsky, A., Bouadjenek, M. R., Hassani, A., & Razzak, I. (2022) · Environment Systems & Decisions · doi:10.1007/s10669-022-09855-1 · Cited by 14
Krishna, M. Y. S., & Gawre, S. K. (2023) · Research Reports on Computer Science · doi:10.37256/rrcs.2320232644 · Cited by 10
“While MLOps is more inclined toward deploying the models into servers, the research focuses on improving the efficiency of models by dynamically changing the parameters precisely through various DevOps practices rather than vaguely changing the hyperparameters inside the models which can be time-consuming [3].”
MLOps critical success factors - A systematic literature review
Mehmood, Y., Sabahat, N., & Ijaz, M. A. (2024) · VFAST Transactions on Software Engineering · doi:10.21015/vtse.v12i1.1747 · Cited by 20
Lestari, M., Iriani, A., & Hendry, H. (2022) · INTENSIF Jurnal Ilmiah Penelitian dan Penerapan Teknologi Sistem Informasi · doi:10.29407/intensif.v6i2.18104 · Cited by 14
“The strategic goal of DevOps is to examine the methods used to improve the quality of services and features in a way that meets user needs [14].”
MethodMLOps in Data Science Projects: A Review
Haertel, C., Staegemann, D. G., Daase, C., Pohl, M., Nahhas, A., & Turowski, K. (2023) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata59044.2023.10386139 · Cited by 16
“Some of the papers advocate for the relevance to initially define the business problem and achieve adequate business understanding (e.g., requirements, success criteria) [12, 16, 32, 36, 53, 62] in a DS/ML-based project.”
BackgroundMLOps-Enabled Security Strategies for Next-Generation Operational Technologies
Ahmad, T., Adnan, M., Rafi, S., Akbar, M. A., & Anwar, A. (2024) · International Conference on Evaluation and Assessment in Software Engineering (EASE) · doi:10.1145/3661167.3661283 · Cited by 17
“Moreover, the deployment and integration of ML models with other systems is crucial to deliver business value, making it a key element of MLOps [17].”
BackgroundInfluentialDevOps Automation Pipeline Deployment with IaC (Infrastructure as Code)
Saxena, A., Singh, S., Prakash, S., Yang, T., & Rathore, R. S. (2024) · IEEE Silchar Subsection Conference (SILCON) · doi:10.1109/silcon63976.2024.10910699 · Cited by 22
“DevOps Cycle [1] [6], [7].”
Haiderzai, M. D., Dakić, P., Stupavský, I., Aleksić, M., & Todorović, V. (2025) · Electronics · doi:10.3390/electronics14020334 · Cited by 14
Toward an Open Source MLOps Architecture
Burgueño-Romero, A. M., Benítez‐Hidalgo, A., Barba-González, C., & Aldana-Montes, J. F. (2024) · IEEE Software · doi:10.1109/ms.2024.3421675 · Cited by 9
Tabuchi, H., Ishitobi, N., Deguchi, H., Nakaniida, Y., Tanaka, H., Akada, M., & Tanabe, M. (2024) · BMJ Quality & Safety · doi:10.1136/bmjqs-2024-018018 · Cited by 8
Bai, E., Song, S. L., Fraser, H., & Ranney, M. L. (2022) · Applied Clinical Informatics · doi:10.1055/s-0041-1740923 · Cited by 9
Suryadevara, M., Rangineni, S., & Venkata, S. (2023) · International Journal of Science and Research (IJSR) · doi:10.21275/sr23719211528 · Cited by 18
“Firstly, several related studies (Karamitsos et (Figalist et al., 2020), or federated learning (Lo et al., 2021).”
BackgroundDevOps Practices Into Machine Learning
Jain, S., & Kumar, P. (2024) · IEEE International Conference on Intelligent Systems, Smart and Green Technologies (ICISSGT) · doi:10.1109/icissgt58904.2024.00029 · Cited by 7
Patrício, L. F. S., Varela, L., & Silveira, Z. D. C. (2025) · Applied Sciences · doi:10.3390/app15073485 · Cited by 14
Intelligent Automation: Leveraging LLMs in DevOps Toolchains
Allam, H. (2024) · International Journal of AI BigData Computational and Management Studies · doi:10.63282/3050-9416.ijaibdcms-v5i4p109 · Cited by 6
Neghawi, E., Wang, Z., Huang, J., & Liu, Y. (2023) · IEEE Annual International Computer Software and Applications Conference (COMPSAC) · doi:10.1109/compsac57700.2023.00185 · Cited by 10
Impact Factors and Best Practices to Improve Effort Estimation Strategies and Practices in DevOps
Meedeniya, D., & Thennakoon, H. (2021) · International Conference on Information Communication and Management (ICICM) · doi:10.1145/3484399.3484401 · Cited by 14
Unlocking the Power of AI/ML in DevSecOps: Strategies and Best Practices
Pakalapati, N., Konidena, B. K., & Mohamed, I. A. (2023) · Journal of Knowledge Learning and Science Technology ISSN: 2959-6386 (online) · doi:10.60087/jklst.vol2.n2.p188 · Cited by 5
On the need to monitor continuous integration practices
Santos, J., da Costa, D. A., McIntosh, S., & Kulesza, U. (2025) · Empirical Software Engineering · doi:10.1007/s10664-025-10682-6 · Cited by 15
“Concerning DevOps, CI serves as a foundational step that allows teams to achieve continuous delivery and continuous deployment [25,31].”
A Survey of Tools, Techniques, and Best Practices: CI/CD Integration in DevOps Workflows
Chandrashekar, P. (2023) · International Journal of Advanced Research in Science Communication and Technology · doi:10.48175/ijarsct-11978v · Cited by 5
Exploring Trade-Offs in MLOps Adoption
John, M. M., Olsson, H. H., Bosch, J., & Gillblad, D. (2023) · Asia-Pacific Software Engineering Conference (APSEC) · doi:10.1109/apsec60848.2023.00047 · Cited by 11
“MLOps share the same foundational concepts as DevOps [17].”
BackgroundAn Exploratory Study on Machine Learning Model Management
Latendresse, J., Abedu, S., Abdellatif, A., & Shihab, E. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3688841 · Cited by 12
“Additionally, we provide a set of actionable recommendations for practitioners and researchers to guide them in their ML model management.”
AN ANALYSIS OF DEVOPS’ IMPACT ON INFORMATION TECHNOLOGY ORGANISATIONS: A CASE STUDY
Mudadi, A., & Lotriet, H. (2023) · The South African Journal of Industrial Engineering · doi:10.7166/34-1-2759 · Cited by 6
“System monitoring helps to understand a software application’s resource utilisation demands, and so makes it possible to implement the appropriate random access memory (RAM) and computer processing units (CPU) that are required for servers to maintain application stability [22].”
BackgroundHow to Support ML End-User Programmers through a Conversational Agent
Garcia, E. J. A., Pimentel, J. F., Feng, Z., Gerosa, M. A., Steinmacher, I., & Sarma, A. (2024) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3597503.3608130 · Cited by 19
“In software engineering, ML models are used to detect bugs, perform code repair, and facilitate DevOps, to name a few applications [21, 41].”
MethodApplying DevOps to Machine Learning Processes: A Systematic Mapping
Matsui, B. M. A., & Goya, D. (2021) · doi:10.5753/eniac.2021.18284 · Cited by 7
“O artigo A9, de [Karamitsos et al. 2020], apresenta práticas, princı́pios e ferramentas de CI/CD para o desenvolvimento e implantação de modelos de ML, com o objetivo de suportar ciclos de feedback rápidos, descobrir dı́vida técnica oculta, aperfeiçoar funções operacionais e a entrega de…”
BackgroundInfluentialHulstijn, J. (2023) · International Conference on Artificial Intelligence and Law · doi:10.1145/3594536.3595122 · Cited by 9
“There are also initiatives to merge machine learning practices and DevOps [41].”
BackgroundMethodInfluentialMLOps: From a Cottage Industry to a Factory Approach
Watson, H. J., & Larson, D. (2024) · International Journal of Business Intelligence Research · doi:10.4018/ijbir.358916 · Cited by 12
Modular oversight methodology: a framework to aid ethical alignment of algorithmic creations
Kyriakou, K., & Otterbacher, J. (2024) · Design Science · doi:10.1017/dsj.2024.23 · Cited by 6
“Continuous delivery is the practice by which a new software version is constantly ready to be tested, evaluated and then released in production (Karamitsos et al. 2020; Symeonidis et al. 2022).”
Machine Learning System Development in Information Systems Development Praxis
Laato, S., Minkkinen, M., Mäntymäki, M., & Dennehy, D. (2024) · Communications of the Association for Information Systems · doi:10.17705/1cais.05406 · Cited by 15
An architectural perspective on MLOps: Structures, processes, tools, and stakeholders
Najafabadi, F. A., Bogner, J., Gerostathopoulos, I., & Lago, P. (2026) · Information and Software Technology · doi:10.1016/j.infsof.2026.108029 · Cited by 4
Cloud Computing Design Patterns for MLOps: Applications to Virtual Power Plants
Subramanya, R., Räisänen, P., Sierla, S., & Vyatkin, V. V. (2023) · Annual Conference of the IEEE Industrial Electronics Society (IECON) · doi:10.1109/iecon51785.2023.10312212 · Cited by 5
“Karamitsos et al. [24] apply DevOps principles to ML applications.”
BackgroundMethodTransforming CS Education with DevOps: Streamlined Assignment Validation and Delivery @ Scale
Deshpande, G., Cheekati, S., Patel, S., Raj, P. M., Singh, M., Pindur, M., Soghyar, N. A., Zhao, B., Babolhavaeji, P., Taher, M., et al. (2024) · ACM Conference on Learning @ Scale (L@S) · doi:10.1145/3657604.3664676 · Cited by 3
“Due to the significant benefits, DevOps is quickly becoming a research priority [14, 15, 17, 39].”
AI-Driven DevOps: A Tool Selection Guide
Sharma, M., Aswathy, C., Ben, M., & Mehrotra, A. (2025) · Lecture notes in electrical engineering · doi:10.1007/978-981-97-8193-5_22 · Cited by 5
Testi, M., Fiorentino, M. C., Ballabio, M., Visani, G., Ciccozzi, M., Frontoni, E., Moccia, S., & Vessio, G. (2025) · Medical & Biological Engineering & Computing · doi:10.1007/s11517-025-03436-5 · Cited by 5
“CI/CD supports faster and more effective model updates [38, 39], while ensuring that production-ready models are robust and error-free [40].”
DevOps Challenges and Practices in Software Engineering
Pandiyavathi, T., & Sivakumar, B. (2023) · Lecture notes in networks and systems · doi:10.1007/978-981-99-1726-6_5 · Cited by 7
Mooduto, A., Rijanto, E., & Pamuji, G. C. (2023) · International Conference on Informatics Engineering, Science & Technology (INCITEST) · doi:10.1109/incitest59455.2023.10397047 · Cited by 3
“CICD concept [7] Dependency-Track monitors component usage across all versions of every portfolio application to proactively identify risk across an organization.”
BackgroundMethodDesign Methods and Processes for ML/DL models
John, M. M. (2021) · doi:10.24834/isbn.9789178771998 · Cited by 4
A MACHINE LEARNING PIPELINE IMPLEMENTATION USING MLOPS AND GITOPS PRINCIPLES
d.o.o, P., Radaković, N., Šenk, I., Romanić, N., & d.o.o, P. (2023) · International Scientific Conference on Industrial Systems (IS) · doi:10.24867/is-2023-t2.1-6_08141 · Cited by 4
A Federated implementation for MLOps framework based on non-intrusive load monitoring
Chen, H., Liu, C.-T., Hsu, H.-Y., & Hsieh, J. (2023) · IEEE Eurasia Conference on IOT, Communication and Engineering (ECICE) · doi:10.1109/ecice59523.2023.10383048 · Cited by 2
“Aiming to apply the matured DevOps concept which has already been used a lot in software development to ML, researchers have developed ML relating it to the current DevOps architecture [8].”
MethodKanita, N. (2024) · Systèmes d information & management · doi:10.3917/sim.234.0007 · Cited by 3
AI-GFA: Applied Framework for Producing Responsible Artificial Intelligence
Iliadis, E. (2024) · International Conference on Information Technology for Social Good (GoodIT) · doi:10.1145/3677525.3678646 · Cited by 2
“• Structured process that aligns with AI developer workflows [3, 4, 29, 30] and simplifies ethical considerations and discussions without requiring extensive time and resources to be invested.”
Efficient Auto Scaling of Pods in Kubernetes: Accelerating Continuous Delivery with KEPTN
Sivaraman, P., Prabaharan, G., Rajasekar, V., & Sarveshwaran, V. (2024) · International Conference on Electronics and Sustainable Communication Systems (ICESC) · doi:10.1109/icesc60852.2024.10689854 · Cited by 2
“Karamitsos et al [10] suggests using DevOps techniques to integrate development and operations environments for machine learning applications.”
MethodAl-Hadrawi, B. K., ALhadrawi, K. K., Ezzerouali, S. A., Al-Hadraawy, S. K., Aldhalmi, H. K., & Alshawabkeh, M. A. (2025) · Journal of Lifestyle and SDGs Review · doi:10.47172/2965-730x.sdgsreview.v5.n03.pe03865 · Cited by 4
“Striking a balance between innovation and safeguarding personal information remains an ongoing challenge (Karamitsos et al ., 2020).”
Cloud-Native Countinous Integration/Continous Deployment (CI/CD) Pipeline
Kumar, S., Bala, E. N., Singh, A. P., & Raj, Y. (2024) · International Conference on Advanced Computing & Communication Technologies (ICACCTech) · doi:10.1109/icacctech65084.2024.00054 · Cited by 2
USING MLOPS FOR DEPLOYMENT OF OPINION MINING MODEL AS A SERVICE FOR SMART CITY APPLICATIONS
Mussina, A., Yedilkhan, D., Alimzhanov, Y., Nugumanova, A., Aubakirov, S., & Mansurova, A. (2025) · Scientific Journal of Astana IT University · doi:10.37943/21cpqx5616 · Cited by 3
“Its goal is to manage the quality of production and operation of machine learning models [4], [5].”
Modernizing CI/CD Pipelines: Migrating from Legacy Tools to GitLab for Enterprise Applications
Koneru, N. M. K. (2021) · International Journal of Science and Research Archive · doi:10.30574/ijsra.2021.1.2.0027 · Cited by 2
Enhancing Devops Infrastructure For Efficient Management Of Microservice Applications
Ekanayaka, E. M. I. M., Thathsarani, J. K. K. H., Karunanayaka, D. S., Kuruwitaarachchi, N., & Skandhakumar, N. (2023) · IEEE International Conference on e-Business Engineering (ICEBE) · doi:10.1109/icebe59045.2023.00035 · Cited by 2
“By [1], highlight the importance of continuous integration (CI) and continuous delivery (CD) in the DevOps framework for automating model retraining in production.”
BackgroundTatineni, S. (2022) · Journal of Economics & Management Research · doi:10.47363/jesmr/2022(3)201 · Cited by 2
Leveraging Gamification for Improved Collaboration in DevOpsPreprint
Mueller, S. (2024) · doi:10.36227/techrxiv.171174564.40878250/v1 · Cited by 2
Sustainable Engineering of Machine Learning-Enabled Systems: A Systematic Mapping StudyPreprint
Chadli, K., Botterweck, G., & Saber, T. (2024) · Research Square · doi:10.21203/rs.3.rs-4694122/v1 · Cited by 8
aiSTROM–A Roadmap for Developing a Successful AI Strategy
Herremans, D. (2021) · IEEE Access · doi:10.1109/access.2021.3127548 · Cited by 41
“Another recent set of best practices is that of MLOps [73], which arose from the fusion of machine learning in DevOps practices [74].”
BackgroundAI-Driven Innovations in Pedagogical and Andragogical Approaches: A Case Study Analysis
Asrifan, A., Shafa, S., Octaberlina, L. R., Ersyad, Z. M., Hermansyah, S., & Amri, N. A. (2024) · Advances in educational technologies and instructional design book series · doi:10.4018/979-8-3693-6130-6.ch009 · Cited by 3
KEENsight: Cloud Based Collaborative Environment for Streamlining Machine Learning Development
Sherpa, L., Attar-Khorasani, S., Sankar, R., Müller-Pfefferkorn, R., & Ghiasvand, S. (2024) · International Conference on Information, Intelligence, Systems & Applications (IISA) · doi:10.1109/iisa62523.2024.10786688 · Cited by 1
“Karamitos et al. introduce DevOps practices in the context of machine learning applications [20], seamlessly integrating development and operational environments.”
Paul, S. P., Khan, S. T., Tasnim, Z., Haque, R., & Islam, M. S. (2026) · Lecture notes in networks and systems · doi:10.1007/978-981-96-8107-5_54 · Cited by 1
Achibat, M. Y., Aqili, N., & Ziti, S. (2026) · Lecture notes in networks and systems · doi:10.1007/978-3-032-07718-9_15 · Cited by 1
Al-Baltah, I. A., Al-Shaibany, N. A., Abdellatief, M., Al-Gawda, M. M., & Al-Sultan, S. Y. (2026) · Information · doi:10.3390/info17020178 · Cited by 1
“Therefore, researchers have attempted to integrate machine learning techniques to optimize DevOps practices [22,23].”
A process model to facilitate AI-based sensor manufacturing optimization
Podder, I., Bub, U., & Winter, R. (2026) · Information Systems and e-Business Management · doi:10.1007/s10257-026-00724-x · Cited by 1
Muchu, M. (2026) · International Conference on Multidisciplinary Innovations for Smart & Sustainable Future (MISSF) · doi:10.1109/missf68264.2026.11522172 · Cited by 1
“Solutions using Flyway and related migration frameworks use rollback mechanisms and solutions based on repeatable, versioned change scripts [16].”
Vänskä, S., Kemell, K., Mikkonen, T., & Abrahamsson, P. (2023) · e-Informatica Software Engineering Journal · doi:10.37190/e-inf240102 · Cited by 8
“Continuous SE builds on agile SE, and agile has been argued to be an important requirement for adopting, e.g., DevOps [8].”
BackgroundMethodContinuous Integration of Neural Networks in Autonomous Systems
Steffen, B., Zohren, J., Pazarci, U., Kullmann, F., & Weißenfels, H. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-49252-5_21 · Cited by 1
“We do this by applying DevOps techniques such as continuous integration to the machine learning process, a workflow that was also presented and evaluated by Karamitsos et al. [4].”
A Case Study on Data Science Processes in an Academia-Industry Collaboration
Silva, S., & de França, B. B. N. (2023) · Brazilian Symposium on Software Quality (SBQS) · doi:10.1145/3629479.3629514 · Cited by 3
“Applying SE techniques can also help with these challenges and improve quality [12].”
BackgroundMachine Learning for Smart School Selection and Admissions
Praveenraj, D. D. W., Victor, M., & Subramani, K. (2023) · International Conference on Computation, Automation and Knowledge Management (ICCAKM) · doi:10.1109/iccakm58659.2023.10449592 · Cited by 1
“Building a scalable and robust system with simple interfaces to data sources, user interfaces, and machine learning models is part of the development process[26].”
BackgroundMannan, M. A., Mustafa, S., Hussain, A., & Hussain, A. (2024) · Journal of Independent Studies and Research - Computing · doi:10.31645/jisrc.24.22.1.4 · Cited by 1
“Data scientists and operations staff are attempting to apply DevOps ideas to their ML systems in organizations in order to enhance value generation and automate the form start to finish life cycle of ML [2].”
BackgroundLi, J., Wan, Q., Ma, W., Fang, X.-T., Yin, C., & Cui, J. (2024) · Transactions on Computer Science and Intelligent Systems Research · doi:10.62051/gc7xf329 · Cited by 1
Metadata
- Authors
- Karamitsos, I., Thabit, S., & Apostolopoulos, C.
- Affiliations
- Rochester Institute of Technology - Dubai (AE)
- University of Strathclyde (GB)
- Corresponding author
- Karamitsos, Ioannis
- Venue
- Information
- Publisher
- MDPI AG
- Published
- 2020-07-13
- Volume
- 11
- Issue
- 7
- Pages
- 363
- ISSN
- 2078-2489 (ISSN-L)
- OpenAlex
- W3041326191
- Cited by
- 140
- References
- 16
- Citations by year
- 2026: 22 · 2025: 18 · 2024: 35 · 2023: 36 · 2022: 16 · 2021: 13
- FWCI
- 15.29
- Type
- journal article
- Language
- en
- Open access
- gold
- dx.doi.org
- Licence
- Locations
- Information (publishedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
- Physical Sciences › Computer Science › Computer Networks and Communications › Software System Performance and Reliability
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Keywords
- DevOps, machine learning, continuous integration, continuous delivery, machine learning deployment, model maintenance, technical debt, feedback loops, value delivery, operational monitoring
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.