Demystifying MLOps and Presenting a Recipe for the Selection of Open-Source Tools
Open accessThe DOI matches its Crossref record; 61 of the 61 references it cites were found in a registry or an index.
Abstract
Nowadays, machine learning projects have become more and more relevant to various real-world use cases. The success of complex Neural Network models depends upon many factors, as the requirement for structured and machine learning-centric project development management arises. Due to the multitude of tools available for different operational phases, responsibilities and requirements become more and more unclear. In this work, Machine Learning Operations (MLOps) technologies and tools for every part of the overall project pipeline, as well as involved roles, are examined and clearly defined. With the focus on the inter-connectivity of specific tools and comparison by well-selected requirements of MLOps, model performance, input data, and system quality metrics are briefly discussed. By identifying aspects of machine learning, which can be reused from project to project, open-source tools which help in specific parts of the pipeline, and possible combinations, an overview of support in MLOps is given. Deep learning has revolutionized the field of Image processing, and building an automated machine learning workflow for object detection is of great interest for many organizations. For this, a simple MLOps workflow for object detection with images is portrayed.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures



Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Packaging models in containers is a commonly reported method of integration, in which the model is accessible through representation state transfer (REST) or remote procedure call (RPC) interfaces [12,16,28,45].”
From Operationalizing machine learning models: a systematic literature review (2022) · cites this paper as [45]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Models packaged in containers are simply run directly as standalone services [16,19,28,29,45].”
From Operationalizing machine learning models: a systematic literature review (2022) · cites this paper as [45]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Polyaxon 9 , MLflow 10 , TFX 11 , ZenML 12 , Flyte 13 , Seldon Core 14 and BentoML 15 are some of the more fully-integrated platforms that exist [10,45], along with the Amazon-exclusive Sage-Maker 16 [9,45].”
From Operationalizing machine learning models: a systematic literature review (2022) · cites this paper as [45]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In productive settings, it is challenging to operate ML manually due to different stacks of software and hardware components and their interplay as well as the selection of both ( [64], [65].”
From Machine Learning Operations (MLOps): Overview, Definition, and Architecture (2023) · cites this paper as [64]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Models are commonly made available for serving predictions through a REST [12,16,26,29,40,45] or RPC [12,28,45] API.”
From Operationalizing machine learning models: a systematic literature review (2022) · cites this paper as [45]
6 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Image quality assessment: from error visibility to structural similarity
Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004) · IEEE Transactions on Image Processing · doi:10.1109/tip.2003.819861 · Cited by 57,892 · cited 1×
“Brief Metric Description SSIM [46] Predict image quality by measuring the structural similarity between two images.”
BackgroundFSIM: A Feature Similarity Index for Image Quality Assessment
Zhang, L., Zhang, L., Mou, X., & Zhang, D. (2011) · IEEE Transactions on Image Processing · doi:10.1109/tip.2011.2109730 · Cited by 5,203
Image information and visual quality
Sheikh, H. R., & Bovik, A. C. (2006) · IEEE Transactions on Image Processing · doi:10.1109/tip.2005.859378 · Cited by 4,104
Bergstra, J., Yamins, D. L. K., & Cox, D. (2013) · Digital Access to Scholarship at Harvard (DASH) (Harvard University) · Cited by 3,163
Most apparent distortion: full-reference image quality assessment and the role of strategy
Larson, E. C., & Chandler, D. M. (2010) · Journal of Electronic Imaging · doi:10.1117/1.3267105 · Cited by 1,953 · cited 1×
“MAD [55] The most apparent distortion metric utilizes contrast sensitivity, local luminance, and contrast masking for high-resolution images.”
BackgroundGradient Magnitude Similarity Deviation: A Highly Efficient Perceptual Image Quality Index
Xue, W., Zhang, L., Mou, X., & Bovik, A. C. (2014) · IEEE Transactions on Image Processing · doi:10.1109/tip.2013.2293423 · Cited by 1,697
Learning under Concept Drift: A Review
Lu, J., Liu, A., Dong, F., Gu, F., Gama, J., & Zhang, G. (2018) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2018.2876857 · Cited by 1,603 · cited 1×
“As in ML, the performance of the model deployed might degrade with time due to specific phenomena such as concept drift, and data drift [31].”
BackgroundAutomated Machine Learning: Methods, Systems, Challenges
(2019) · The Springer Series on Challenges in Machine Learning · doi:10.1007/978-3-030-05318-5 · Cited by 1,485
Information content weighting for perceptual image quality assessment
Wang, Z., & Li, Q. (2011) · IEEE Transactions on Image Processing · doi:10.1109/tip.2010.2092435 · Cited by 1,304
VSI: A Visual Saliency-Induced Index for Perceptual Image Quality Assessment
Zhang, L., Shen, Y., & Li, H. (2014) · IEEE Transactions on Image Processing · doi:10.1109/tip.2014.2346028 · Cited by 1,029 · cited 1×
“VSI [57] Visual Saliency-Induced Index uses ‘saliency map’ as a quality feature and weighting function to characterize the importance of local regions.”
Research works printed but not linked: 6
- 2. Goyal, A. Machine Learning Operations. In International Journal of Information Technology Insights & Transformations [ISSN: 2581-5172 (Online)] ; Eureka Journals: Pune, India 2020; Volume 4.
- 10. Zhao, Y. Machine Learning in Production: A Literature Review. 2021. Available online: https://staff.fnwi.uva.nl/a.s.z.belloum/ LiteratureStudies/Reports/2021-LiteratureStudy-report-Yizhen.pdf (accessed on 27 July 2021).
- 13. Sureddy, M.R.; Yallamula, P. A Framework for Monitoring Data Warehousing Applications. Int. Res. J. Eng. Technol. 2020 , 7 , 7023-7029.
- 22. Zöller, M.A.; Huber, M.F. Survey on automated machine learning. arXiv 2019 , arXiv:1904.12054. · arXiv
- 30. Treveil, M.; Omont, N.; Stenac, C.; Lefevre, K.; Phan, D.; Zentici, J.; Lavoillotte, A.; Miyazaki, M.; Heidmann, L. Introducing MLOps ; O'Reilly Media: Sebastopol, CA, USA, 2020.
- 42. Gudivada, V.; Apon, A.; Ding, J. Data quality considerations for big data and machine learning: Going beyond data cleaning and transformations. Int. J. Adv. Softw. 2017 , 10 , 1-20.
Cited by
Items 101–138 of 138
Cychnerski, J., & Dziubich, T. (2024) · Proceedings of the International Conference on Information Systems Development · doi:10.62036/isd.2024.18 · Cited by 1
“…from discussions surrounding optimal methodologies for machine learning across various domains such as manufacturing [7, 10], physical systems [14], and health in-formatics [6, 20, 19], we have opted to introduce best practices tailored for conducting projects centered on the analysis of…”
Pistor, N. (2023) · DiVA (Linnaeus University) · Cited by 1
Model Operationalization at Edge Devices
Kwatra, S., Mangla, U., & Srivatsa, M. (2023) · Springer eBooks · doi:10.1007/978-3-031-40787-1_12
Enhancing Collaboration and Agility in Data-Centric AI Projects
Stieler, F., & Bauer, B. (2024) · Communications in computer and information science · doi:10.1007/978-3-031-64182-4_15 · Cited by 1
Towards continuous development of MLOps practices
John, M. M. (2024) · doi:10.24834/isbn.9789178775637
الجمل, ز. ش. ح. ا., الفنجري, ل., & Fahmy, H. (2025) · النشرة المعلوماتية في الحاسبات والمعلومات · doi:10.21608/fcihib.2025.261807.1105
Towards more Credible Agent-Based Models for Policy Support
Johansson, E. (2025) · doi:10.24834/isbn.9789178775750
METHOD OF APPLYING MACHINE LEARNING TO ENHANCE THE EFFICIENCY OF DEVOPS PROCESSES
PRAVORSKA, N. (2024) · Herald of Khmelnytskyi National University Technical sciences · doi:10.31891/2307-5732-2024-343-6-68
Habtemariam, S., Yihdego, D., & Lambert, E. (2025) · Transactions on Engineering and Computing Sciences · doi:10.14738/tecs.1302.18536
Research and Development of Kubernetes Operator for Machine Learning Pipelines
Trapezin, A., & Filianin, I. (2025) · International Russian Smart Industry Conference (SmartIndustryCon) · doi:10.1109/smartindustrycon65166.2025.10985995
“One of the key tools in this area is the MLflow platform [5-6].”
MLOps for Medical Imaging: A Cloud-Edge architecture for Experiment Tracking and Model Evaluation
Saim, Akram, S. U., Harjula, E., Lovén, L., & Mehmood, H. (2025) · IEEE International Conference on Network Protocols (ICNP) · doi:10.1109/icnp65844.2025.11192460
“…design validates the effectiveness of plugin-based extensions for specialized MLOps applications, demonstrating how existing platforms can be enhanced with domain-specific capabilities without compromising core functionality or requiring extensive modifications to proven infrastructure [15].”
Scalable AI Deployment for Healthcare: A Kubernetes-Based Pipeline for Real-Time Medical Predictions
Babu, A., Athiramol, S., Yousaf, S., & Elayidom, M. S. (2025) · International Conference on Metaverse and Current Trends in Computing (ICMCTC) · doi:10.1109/icmctc62214.2025.11196190
“This paper focuses on the implementation of Kubernetes-based technologies [1], specifically KServe and Kubeflow, for streamlining the deployment and management of deep learning models in medical imaging diagnosis.”
Toward the Reliable Deployment of Computer Vision Pipelines by Leveraging MLOps and Explainable AI
Nerantzis, E., Symeonidis, G., Papageorgiou, A. V., & Papakostas, G. A. (2025) · Lecture notes in networks and systems · doi:10.1007/978-981-96-8694-0_9
Agile Methoden zur Einführung von KI im öffentlichen Sektor
Pechtor, V. (2025) · Interdisziplinäre Organisations- und Verwaltungsforschung · doi:10.1007/978-3-658-48296-1_7
From Laboratory State into Production: Automating Hyperspectral Image Classification through MLOps
Herrmann, L., Griesel, A., Mossner, C., Schwarze, J., & Atzmueller, M. (2025) · IEEE International Conference on Tools with Artificial Intelligence (ICTAI) · doi:10.1109/ictai66417.2025.00076
“The lack of versioning artifacts such as source code, models, and data sets contributes to the difficulty of collaboration within projects [3], [4].”
InfluentialWerle, F. (2023) · reposiTUm (TU Wien) · doi:10.34726/hss.2023.117562
Peñafiel Mora, D. M., & Seaman Mora, D. A. (2023) · Repositorio Institucional (Universidad de Cuenca)
Benefits of machine learning in operational management systems in the social and healthcare sectors
Huttunen, J. (2024) · LUTPub (LUT University) · Cited by 3
Seganfredo, H. (2026) · Portuguese National Funding Agency for Science, Research and Technology (RCAAP Project by FCT)
Waseem, H. (2024) · Tampere University Institutional Repository (Tampere University)
MLOps Architecture for Machine Vision: A Practical Architecture for Industry Application
Linna, M. (2025) · Tampere University Institutional Repository (Tampere University)
Pipeline Optimization in MLOps: A Review of Approaches and Current Challenges
Benhirt, B. E., Lachgar, M., & Errattahi, R. (2025) · International Conference on Networking, Information Systems & Security (NISS) · doi:10.1109/niss66502.2025.00024
Esteban, A., & Calvache, C. D. J. P. (2026) · Inge CUC · doi:10.17981/ingecuc.22.1.2026.03
CLASSIFICATION OF ARTIFICIAL INTELLIGENCE TOOLING IN ARCHITECTURE. PROCESS-CYCLE DIMENSION
Вергунова, Н. С. (2026) · Municipal economy of cities · doi:10.33042/3083-6727-2026-1-196-125-132
Weigell, B., Stieler, F., & Bauer, B. (2025) · International Conference on Cloud and Big Data Computing (ICCBDC) · doi:10.1109/iccbdc67784.2025.00016
“Although helpful frameworks and selection guides exist, such as [5], [6], they typically focus on pipeline-level orchestration and often neglect fundamental aspects like the provisioning and management of the underlying infrastructure.”
InfluentialManaging Legacy Infrastructure with Modern Automation Toolchains
Siddiqui, N. (2026) · International Journal on Science and Technology · doi:10.71097/ijsat.v17.i2.11330
“As Ruf et al. (Philipp Ruf et al., 2021) explain in MLOps, detailed monitoring is key for managing complex systems, and this idea also applies to infrastructure automation.”
Challenges and ethical considerations in implementing Machine Learning
Galal, M. K., & Said, Z. (2026) · Elsevier eBooks · doi:10.1016/b978-0-443-33943-1.00003-4
Daoud, C., Azar, D., Urtado, C., & Vauttier, S. (2026) · Future Generation Computer Systems · doi:10.1016/j.future.2026.108700
From MLOps to LLMOps: A Position on Engineering Discipline for AI-Native Applications
Mandadi, A. R. (2024) · International Journal of Scientific Research and Modern Technology. · doi:10.38124/ijsrmt.v3i4.1566
Integrating MLOps for Monitoring, Drift Detection, and Adaptive Retraining in Sensor Systems
Heriyani, N., & Merlina, N. (2026) · Computer Science (CO-SCIENCE) · doi:10.31294/co-science.v6i2.12409 · Cited by 1
KAYSAL, K. (2026) · Scientific Reports · doi:10.1038/s41598-026-62648-3
Developing Robust and Reproducible Machine Learning Systems
Polychronou, I., Nikolaou, G., & Patrikakis, C. Z. (2026) · IEEE Annual International Computer Software and Applications Conference (COMPSAC) · doi:10.1109/compsac69091.2026.00142
“Traditional ML workflows present unique challenges that differ from conventional software development, often hindering the efficiency and effectiveness of ML systems [4].”
An Iterative Framework for AI Adoption in the Public Sector: Insights from a Swiss Case Study
Pechtor, V. (2026) · JeDEM - eJournal of eDemocracy and Open Government · doi:10.29379/jedem.v18i1.1158
Home cage monitoring of group-housed animals in highly enriched enclosures
Lewis, K., Ciminelli, G., Andres, T., Stupples, K., & Witham, C. L. (2026) · Current Research in Behavioral Sciences · doi:10.1016/j.crbeha.2026.100229
MLOps para Empresas: Elaboração e Avaliação de um Guia de Boas Práticas
Ruseler, K. V. (2026) · DSpace (Federal University of Santa Catarina)
An Integrated Container Monitoring Model Using Machine Learning Operations
Elgamal, Z. S. (2024) · International Journal of Intelligent Systems and Applications in Engineering
Manoppo, F. G. C., Kaunang, H. G., & Waworuntu, B. J. (2026) · JOURNAL OF SCIENCE AND SOCIAL RESEARCH · doi:10.54314/rkskc803
Demirsoy, H. B., & Akcayol, M. A. (2026) · International Artificial Intelligence and Data Processing Symposium (IDAP) · doi:10.1109/idap71869.2026.11728053
“The responsible AI engineering model shows how governance principles can be implemented through design and implementation [12], [13], [15].”
Metadata
- Authors
- Ruf, P., Madan, M., Reich, C., & Ould-Abdeslam, D.
- Affiliations
- Furtwangen University (DE)
- Institut de Recherche en Informatique, Mathématiques, Automatique et Signal (IRIMAS), Université de Haute-Alsace, 61 Rue Albert Camus, 68093 Mulhouse, France
- Corresponding author
- Reich, Christoph
- Venue
- Applied Sciences
- Publisher
- MDPI AG
- Published
- 2021-09-23
- Volume
- 11
- Issue
- 19
- Pages
- 8861
- ISSN
- 2076-3417 (ISSN-L)
- OpenAlex
- W3202461888
- Cited by
- 112
- References
- 50
- Citations by year
- 2026: 11 · 2025: 22 · 2024: 33 · 2023: 28 · 2022: 18
- FWCI
- 17.86
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Applied Sciences (publishedVersion, cc-by)
- Repository of Futwangen University of Applied Sciences (Furtwangen University) (submittedVersion, cc-by)
- HAL (submittedVersion, other-oa)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
- Social Sciences › Decision Sciences › Information Systems and Management › Scientific Computing and Data Management
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
- Keywords
- MLOps, open source tools, machine learning pipelines, tool selection, object detection, deep learning, image processing, model evaluation metrics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.