Applying DevOps Practices of Continuous Automation for Machine Learning
Open accessThe DOI matches its Crossref record; 16 of the 16 references it cites were found in a registry or an index.
Abstract
This paper proposes DevOps practices for machine learning application, integrating both the development and operation environment seamlessly. The machine learning processes of development and deployment during the experimentation phase may seem easy. However, if not carefully designed, deploying and using such models may lead to a complex, time-consuming approaches which may require significant and costly efforts for maintenance, improvement, and monitoring. This paper presents how to apply continuous integration (CI) and continuous delivery (CD) principles, practices, and tools so as to minimize waste, support rapid feedback loops, explore the hidden technical debt, improve value delivery and maintenance, and improve operational functions for real-world machine learning applications.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures

![Figure 2 CRISP-DM methodology (Source: [26]).](https://figures.cochat.ai/paper-figures/a12adec26fd648cb876870d34b76a13d/applying-devops-practices-of-continuous-automation-for-machine-learning-figure-2.png)
![Figure 3 SEMMA methodology (Source: Adapted [27]). Figure 3. SEMMAmethodology (Source: Adapted [27]).](https://figures.cochat.ai/paper-figures/81d282ad0390445ba6f400b6e39f28d6/applying-devops-practices-of-continuous-automation-for-machine-learning-figure-3.png)
![Figure 4 TDSP methodology (Source [28]). Figure 4. TDSP methodology (Source [28]).](https://figures.cochat.ai/paper-figures/45b1c2a73786479e8034398452b468ee/applying-devops-practices-of-continuous-automation-for-machine-learning-figure-4.png)


Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Firstly, several related studies [18, 15, 16, F42, 6, 9, 21, 22, 26] are based on a limited amount of identified primary sources, not covering relevant insights from the wide range of existing literature.”
From The pipeline for the continuous development of artificial intelligence models—Current state of research and practice (2023) · cites this paper as [18]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Additionally, monitoring of the ML infrastructure, CI/CD, and orchestration are required[7],[23],[24],[28],[32],[33], [50], [α, ζ, η, θ].”
From Machine Learning Operations (MLOps): Overview, Definition, and Architecture (2023) · cites this paper as [24]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
(2015) · neural information processing systems · Cited by 912
Continuous software engineering: A roadmap and agenda
Fitzgerald, B., & Stol, K.-J. (2017) · Journal of Systems and Software · doi:10.1016/j.jss.2015.06.063 · Cited by 723 · cited 3×
“The automated CI/CD system allows the data scientists to rapidly explore new ideas around feature engineering, model architecture, and hyperparameters.”
BackgroundInfluentialEbert, C., Gallardo, G., Hernantes, J., & Serrano, N. (2016) · IEEE Software · doi:10.1109/ms.2016.68 · Cited by 504
Chacon, S., & Straub, B. (2014) · Apress eBooks · doi:10.1007/978-1-4842-0076-6 · Cited by 292
Continuous Delivery: Huge Benefits, but Challenges Too
Chen, L. (2015) · IEEE Software · doi:10.1109/ms.2015.27 · Cited by 261 · cited 1×
“According to [11,12], this practice offers several benefits such as reduced deployment risk, lower costs, and getting user feedback faster.”
BackgroundA qualitative study of DevOps usage in practice
Erich, F. M. A., Amrit, C., & Daneva, M. (2017) · Journal of Software: Evolution and Process · doi:10.1002/smr.1885 · Cited by 203 · cited 1×
“In this study, the TDSP methodology is selected.”
MethodThe highways and country roads to continuous deployment
Leppanen, M., Makinen, S., Pagels, M., Eloranta, V.-P., Itkonen, J., Mantyla, M. V., & Mannisto, T. (2015) · IEEE Software · doi:10.1109/ms.2015.50 · Cited by 150 · cited 1×
“This practice includes automated software building and testing [7].”
MethodUnderstanding DevOps & bridging the gap from continuous integration to continuous delivery
Virmani, M. (2015) · Fifth International Conference on the Innovative Computing Technology (INTECH 2015) · doi:10.1109/intech.2015.7173368 · Cited by 132 · cited 2×
“In addition, several researchers (Karamitsos [21], Virmani, [22]; Erich [23]; Lwakatare [24]) have agreed that Agile transformation is essential to improve the efficiency of the companies to optimize the lifecycle delivery, to break the gap and to create a continuous feedback loop between the business users and develo…”
BackgroundRelationship of DevOps to Agile, Lean and Continuous Deployment
Lwakatare, L. E., Kuvaja, P., & Oivo, M. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-49094-6_27 · Cited by 122 · cited 1×
“In this study, the TDSP methodology is selected.”
MethodContinuous Software Engineering
Bosch, J. (2014) · Springer eBooks · doi:10.1007/978-3-319-11283-1 · Cited by 109
Research works printed but not linked: 3
- 25. Forsgren, N.; Humble, J.; Gene, K. The Science of Lean Software and DevOps: Building and Scaling High Performing Technology Organizations ; IT Revolution Press: Portland, OR, USA, 2018.
- 26. Shearer, D. The CRISP-DM Model: The new Blueprint for Data Mining. J. Data Wareh. 2000 , 5 , 13-18.
- 29. Humble, J.; Farley, D. Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation ; Pearson Education, Inc.: Boston, MA, USA, 2011.
Cited by
Items 101–139 of 139
Subramaniam, K., Kumar, S., Mishra, A., Bhandari, A., Ppallan, J. M., & Chandrasekaran, G. (2024) · IEEE Access · doi:10.1109/access.2024.3472498 · Cited by 1
Machine Learning Operations: A Mapping Study
Chakraborty, A., Das, S., & Gary, K. A. (2025) · Communications in computer and information science · doi:10.1007/978-3-031-86644-9_1 · Cited by 8
Evolution of the continuous integration of neural networks in autonomous systems
Steffen, B., Sondermann, L., & Schütz, L. (2025) · International Journal on Software Tools for Technology Transfer · doi:10.1007/s10009-025-00799-w · Cited by 1
“We do this by applying DevOps techniques such as continuous integration to the machine learning process, a workflow that was also presented and evaluated by Karamit-sos et al. [5].”
Evaluation of the Feasibility of Parametric Estimation in Devops Continuous Planning
Tripathi, D. R. (2021) · International Journal for Research in Applied Science and Engineering Technology · doi:10.22214/ijraset.2021.38008 · Cited by 1
Mary, I. M., & Karuppasamy, K. (2023) · Computer Systems Science and Engineering · doi:10.32604/csse.2023.031114 · Cited by 2
PIPELINE-GENERATED CONTINUOUS INTEGRATION AND DEPLOYMENT METHOD FOR AGILE SOFTWARE DEVELOPMENTPreprint
Bhardwaj, A. K., Rangineni, S., & Reddi, L. T. (2023) · doi:10.36227/techrxiv.23682072 · Cited by 1
Festa, G., Giordano, G., Pontillo, V., Penta, M. D., Tamburri, D. A., & Palomba, F. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5393703 · Cited by 1
Rodriguez, B., & Cedeño-Moreno, D. (2023) · I+D Tecnológico · doi:10.33412/idt.v19.1.3783
PIPELINE-GENERATED CONTINUOUS INTEGRATION AND DEPLOYMENT METHOD FOR AGILE SOFTWARE DEVELOPMENTPreprint
Bhardwaj, A. K., Rangineni, S., & Reddi, L. T. (2023) · doi:10.36227/techrxiv.23682072.v1
PIPELINE-GENERATED CONTINUOUS INTEGRATION AND DEPLOYMENT METHOD FOR AGILE SOFTWARE DEVELOPMENTPreprint
Bhardwaj, A. K., Rangineni, S., & Reddi, L. T. (2023) · doi:10.36227/techrxiv.23682072.v2
Machine Learning on Insurance Premium Prediction
Jesus, R. M., Brito, M. A., & Duarte, D. N. (2023) · International Conference on Computer Technology Applications (ICCTA) · doi:10.1145/3605423.3605450
“The CD pipeline allows to launch small updates in a steadily and frequently way, without any human intervention [42].”
BackgroundPIPELINE-GENERATED CONTINUOUS INTEGRATION AND DEPLOYMENT METHOD FOR AGILE SOFTWARE DEVELOPMENTPreprint
Bhardwaj, A. K., Rangineni, S., & Reddi, L. T. (2023) · doi:10.36227/techrxiv.23682072.v3
Unlocking the Potential of AI/ML in DevSecOps: Effective Strategies and Optimal Practices
Camacho, N. G. (2024) · Journal of Artificial Intelligence General science (JAIGS) · doi:10.60087/jaigs.v2i1.99
Enhancing Collaboration and Agility in Data-Centric AI Projects
Stieler, F., & Bauer, B. (2024) · Communications in computer and information science · doi:10.1007/978-3-031-64182-4_15 · Cited by 1
Cychnerski, J., & Dziubich, T. (2024) · Proceedings of the International Conference on Information Systems Development · doi:10.62036/isd.2024.18 · Cited by 1
“Treating both fields separately, numerous publications on best practices can be found in the literature, e.g. [9, 21].”
Fedoryshyn, B. (2024) · Вісник Черкаського державного технологічного університету · doi:10.62660/bcstu/2.2024.57 · Cited by 3
الجمل, ز. ش. ح. ا., الفنجري, ل., & Fahmy, H. (2025) · النشرة المعلوماتية في الحاسبات والمعلومات · doi:10.21608/fcihib.2025.261807.1105
Bahuleyan, H., Puzikov, Y., Koriagin, E., Lasserre, J., Weffer, R., & Shirvany, R. (2024) · International Conference on Software and Information Engineering (ICSIE) · doi:10.1145/3708635.3708636
“On the operationalization side, numerous studies have explored topics such as software engineering practices for ML, deployment challenges and MLOps [1, 19].”
AI as the Co-creator: Xamun’s vision for the new software renaissance
Wadehra, S., & Anand, A. (2025) · Journal of Information Technology Case and Application Research · doi:10.1080/15228053.2025.2525017
(2025) · Journal on Electronic and Automation Engineering · doi:10.46632/jeae/2/4/8
Study of Effective Software Project Management Practices through DevOps Adoption
Ganeshkumar, D., & Shah, S. R. (2025) · International Journal of Applied Sciences & Development · doi:10.37394/232029.2025.4.28
Wurzer, A., Fritz, J., & Cinar, O. (2026) · HMD Praxis der Wirtschaftsinformatik · doi:10.1365/s40702-025-01233-0
“Die DevOps-Pipeline umfasst dabei Praktiken und Werkzeuge, die die gesamte Bereitstellung, von der Codeentwicklung über Build, Test und Deployment bis hin zu Betrieb, Monitoring und kontinuierlichem Feedback, automatisieren (Karamitsos et al. 2020).”
Ahmed, K. R., Islam, M. S., Khan, M. T., Mehbuba, M. S., Apu, B. D., Noyeem, M. M. I., & Goyal, S. K. (2025) · International Symposium on Advanced Electrical and Communication Technologies (ISAECT) · doi:10.1109/isaect68904.2025.11318824
Festa, G., Giordano, G., Pontillo, V., Penta, M. D., Tamburri, D. A., & Palomba, F. (2026) · Information and Software Technology · doi:10.1016/j.infsof.2025.108009 · Cited by 2
Fariñas, A. E., Paz, C. R. L., Llanes, R. S., & Moreno, A. S. P. (2026) · Lecture notes in networks and systems · doi:10.1007/978-3-031-96157-1_47
Machine Learning Operations on ZYNQ FPGA Board for Real-Time Face Recognition
Kouach, B., Mekhfioui, M., & Gouri, R. E. (2026) · Applied System Innovation · doi:10.3390/asi9040071
Dehury, C. K., & Srirama, S. N. (2026) · Elsevier eBooks · doi:10.1016/b978-0-443-45032-7.00011-0
Weigell, B., Stieler, F., & Bauer, B. (2025) · International Conference on Cloud and Big Data Computing (ICCBDC) · doi:10.1109/iccbdc67784.2025.00016
“In MLOps, CD practices are commonly employed to deploy the entire ML pipeline [9], [23], [29], [31] and to integrate the trained models into the overall software architecture [23], [24], [29], [31], [32], reducing deployment time and increasing reliability [32].”
InfluentialBoinpally, D. (2026) · IEEE International Conference on Emerging Computing and Intelligent Technologies (ICoECIT) · doi:10.1109/icoecit68303.2026.11497432 · Cited by 1
“Recent developments in Generative Artificial Intelligence have shown good performance in synthesizing structured, meaningful code and configuration artifacts out of natural language instructions [8].”
Assessing the Impact of AI-Augmented DevSecOps on Lead Time in Agile Release Management
Gunawan, J. A., Singgih, M. L., & Ginardi, R. V. H. (2026) · F1000Research · doi:10.12688/f1000research.178067.1 · Cited by 1
Kumar, S. S. S., Garapati, R. S., Segireddy, A. R., Kalisetty, S., Inala, R., & Nagabhyru, K. C. (2026) · IEEE International Conference for Convergence in Computing Technology (I3CTCON) · doi:10.1109/i3ctcon68242.2026.11507219
CONCEPTUAL MODEL FOR THE USE OF DEVOPS AND AI TOOLS IN AGILE PROJECTSPreprint
Feitosa, D. N. M., Penha, R., Filho, J. R. D. C., Galvão, G. S., & da Silva, V. F. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6956551
Eraganeni, S. R. (2026) · International Journal of Computational and Experimental Science and Engineering · doi:10.22399/ijcesen.5340 · Cited by 1
Daoud, C., Azar, D., Urtado, C., & Vauttier, S. (2026) · Future Generation Computer Systems · doi:10.1016/j.future.2026.108700
Machine Learning Lifecycle: A Survey
Kosmas, I., Papadopoulos, T., & Michalakelis, C. (2026) · AppliedMath · doi:10.3390/appliedmath6070113
“Blockchain-backed validation ledgers represent a compelling architectural pattern to guarantee immutable data tracking across decentralized pipelines [17], provided that latency constraints are systematically controlled.”
Evaluating DevOps, DevSecOps, MLOps and CloudOps–Results from an Adapted MLR
Yakushenko, A., McCann, R., Hughes, C., McElligott, S., Yılmaz, M., MacMahon, S. T., Mccarren, A., & McCarren, A. L. (2026) · Communications in computer and information science · doi:10.1007/978-3-032-36813-3_6
Gholami, M., Zamani, B., & Ghahfarokhi, B. S. (2026) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2026.116086
Varma, D., Parmar, R., Bhat, S., & Dutta, P. (2026) · Future Generation Computer Systems · doi:10.1016/j.future.2026.108807
Design and implementation of MLOps-based healthcare pipelines using the Cybersecurity Framework
Robustelli, A., Marfoglia, A., D’Errico, C., Mellone, S., Carbonaro, A., & Chesani, F. (2026) · Future Generation Computer Systems · doi:10.1016/j.future.2026.108857
Metadata
- Authors
- Karamitsos, I., Thabit, S., & Apostolopoulos, C.
- Affiliations
- Rochester Institute of Technology - Dubai (AE)
- University of Strathclyde (GB)
- Corresponding author
- Karamitsos, Ioannis
- Venue
- Information
- Publisher
- MDPI AG
- Published
- 2020-07-13
- Volume
- 11
- Issue
- 7
- Pages
- 363
- ISSN
- 2078-2489 (ISSN-L)
- OpenAlex
- W3041326191
- Cited by
- 140
- References
- 16
- Citations by year
- 2026: 22 · 2025: 18 · 2024: 35 · 2023: 36 · 2022: 16 · 2021: 13
- FWCI
- 15.29
- Type
- journal article
- Language
- en
- Open access
- gold
- dx.doi.org
- Licence
- Locations
- Information (publishedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
- Physical Sciences › Computer Science › Computer Networks and Communications › Software System Performance and Reliability
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Keywords
- DevOps, machine learning, continuous integration, continuous delivery, machine learning deployment, model maintenance, technical debt, feedback loops, value delivery, operational monitoring
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.