Applying DevOps Practices of Continuous Automation for Machine Learning
Open accessThe DOI matches its Crossref record; 16 of the 16 references it cites were found in a registry or an index.
Abstract
This paper proposes DevOps practices for machine learning application, integrating both the development and operation environment seamlessly. The machine learning processes of development and deployment during the experimentation phase may seem easy. However, if not carefully designed, deploying and using such models may lead to a complex, time-consuming approaches which may require significant and costly efforts for maintenance, improvement, and monitoring. This paper presents how to apply continuous integration (CI) and continuous delivery (CD) principles, practices, and tools so as to minimize waste, support rapid feedback loops, explore the hidden technical debt, improve value delivery and maintenance, and improve operational functions for real-world machine learning applications.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures

![Figure 2 CRISP-DM methodology (Source: [26]).](https://figures.cochat.ai/paper-figures/a12adec26fd648cb876870d34b76a13d/applying-devops-practices-of-continuous-automation-for-machine-learning-figure-2.png)
![Figure 3 SEMMA methodology (Source: Adapted [27]). Figure 3. SEMMAmethodology (Source: Adapted [27]).](https://figures.cochat.ai/paper-figures/81d282ad0390445ba6f400b6e39f28d6/applying-devops-practices-of-continuous-automation-for-machine-learning-figure-3.png)
![Figure 4 TDSP methodology (Source [28]). Figure 4. TDSP methodology (Source [28]).](https://figures.cochat.ai/paper-figures/45b1c2a73786479e8034398452b468ee/applying-devops-practices-of-continuous-automation-for-machine-learning-figure-4.png)


Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Firstly, several related studies [18, 15, 16, F42, 6, 9, 21, 22, 26] are based on a limited amount of identified primary sources, not covering relevant insights from the wide range of existing literature.”
From The pipeline for the continuous development of artificial intelligence models—Current state of research and practice (2023) · cites this paper as [18]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Additionally, monitoring of the ML infrastructure, CI/CD, and orchestration are required[7],[23],[24],[28],[32],[33], [50], [α, ζ, η, θ].”
From Machine Learning Operations (MLOps): Overview, Definition, and Architecture (2023) · cites this paper as [24]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
(2015) · neural information processing systems · Cited by 912
Continuous software engineering: A roadmap and agenda
Fitzgerald, B., & Stol, K.-J. (2017) · Journal of Systems and Software · doi:10.1016/j.jss.2015.06.063 · Cited by 723
Ebert, C., Gallardo, G., Hernantes, J., & Serrano, N. (2016) · IEEE Software · doi:10.1109/ms.2016.68 · Cited by 504
Chacon, S., & Straub, B. (2014) · Apress eBooks · doi:10.1007/978-1-4842-0076-6 · Cited by 292
Continuous Delivery: Huge Benefits, but Challenges Too
Chen, L. (2015) · IEEE Software · doi:10.1109/ms.2015.27 · Cited by 261
A qualitative study of DevOps usage in practice
Erich, F. M. A., Amrit, C., & Daneva, M. (2017) · Journal of Software: Evolution and Process · doi:10.1002/smr.1885 · Cited by 203
The highways and country roads to continuous deployment
Leppanen, M., Makinen, S., Pagels, M., Eloranta, V.-P., Itkonen, J., Mantyla, M. V., & Mannisto, T. (2015) · IEEE Software · doi:10.1109/ms.2015.50 · Cited by 150
Understanding DevOps & bridging the gap from continuous integration to continuous delivery
Virmani, M. (2015) · Fifth International Conference on the Innovative Computing Technology (INTECH 2015) · doi:10.1109/intech.2015.7173368 · Cited by 132
Relationship of DevOps to Agile, Lean and Continuous Deployment
Lwakatare, L. E., Kuvaja, P., & Oivo, M. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-49094-6_27 · Cited by 121
Continuous Software Engineering
Bosch, J. (2014) · Springer eBooks · doi:10.1007/978-3-319-11283-1 · Cited by 109
Research works printed but not linked: 3
- Forsgren, N., Humble, J., and Gene, K. (2018). The Science of Lean Software and DevOps: Building and Scaling High Performing Technology Organizations, IT Revolution Press.
- Shearer, 2000, The CRISP-DM Model: The new Blueprint for Data Mining, J. Data Wareh., 5, 13
- Humble, J., and Farley, D. (2011). Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation, Pearson Education, Inc.
Cited by
Machine Learning Operations (MLOps): Overview, Definition, and Architecture
Kreuzberger, D., Kühl, N., & Hirschl, S. (2023) · IEEE Access · doi:10.1109/access.2023.3262138 · Cited by 766
“The solution architect designs the architecture and defines the technologies to be used, following a thorough evaluation [17,27] [α, ζ].”
BackgroundMethodInfluentialHow to explain AI systems to end users: a systematic literature review and research agenda
Laato, S., Tiainen, M., Najmul Islam, A. K. M., & Mäntymäki, M. (2022) · Internet Research · doi:10.1108/intr-08-2021-0600 · Cited by 155
MLOps - Definitions, Tools and Challenges
Symeonidis, G., Nerantzis, E., Kazakis, A., & Papakostas, G. A. (2022) · IEEE Annual Computing and Communication Workshop and Conference (CCWC) · doi:10.1109/ccwc54503.2022.9720902 · Cited by 180
Steidl, M., Felderer, M., & Ramler, R. (2023) · Journal of Systems and Software · doi:10.1016/j.jss.2023.111615 · Cited by 126
Towards MLOps: A Framework and Maturity Model
John, M. M., Olsson, H. H., & Bosch, J. (2021) · 2021 47th Euromicro Conference on Software Engineering and Advanced Applications (SEAA) · doi:10.1109/seaa53835.2021.00050 · Cited by 130
Metadata
- Authors
- Karamitsos, I., Thabit, S., & Apostolopoulos, C.
- Affiliations
- Rochester Institute of Technology - Dubai (AE)
- University of Strathclyde (GB)
- Corresponding author
- Karamitsos, Ioannis
- Venue
- Information
- Publisher
- MDPI AG
- Published
- 2020-07-13
- Volume
- 11
- Issue
- 7
- Pages
- 363
- ISSN
- 2078-2489 (ISSN-L)
- OpenAlex
- W3041326191
- Cited by
- 140
- References
- 16
- Citations by year
- 2026: 22 · 2025: 18 · 2024: 35 · 2023: 36 · 2022: 16 · 2021: 13
- FWCI
- 15.29
- Type
- journal article
- Language
- en
- Open access
- gold
- dx.doi.org
- Licence
- Locations
- Information (publishedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
- Physical Sciences › Computer Science › Computer Networks and Communications › Software System Performance and Reliability
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Keywords
- DevOps, machine learning, continuous integration, continuous delivery, machine learning deployment, model maintenance, technical debt, feedback loops, value delivery, operational monitoring
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Applying DevOps Practices of Continuous Automation for Machine Learning. https://app.cochat.ai/papers/10.3390/info11070363
- The paper
- Karamitsos, I., Thabit, S., & Apostolopoulos, C. (2020). Applying DevOps Practices of Continuous Automation for Machine Learning. Information. https://doi.org/10.3390/info11070363
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.