Towards Trustworthy Machine Learning in Production: An Overview of the Robustness in MLOps Approach
Open accessThe DOI matches its Crossref record; 178 of the 178 references it cites were found in a registry or an index.
Abstract
Artificial intelligence (AI), and especially its sub-field of Machine Learning (ML), are impacting the daily lives of everyone with their ubiquitous applications. In recent years, AI researchers and practitioners have introduced principles and guidelines to build systems that make reliable and trustworthy decisions. From a practical perspective, conventional ML systems process historical data to extract the features that are consequently used to train ML models that perform the desired task. However, in practice, a fundamental challenge arises when the system needs to be operationalized and deployed to evolve and operate in real-life environments continuously. To address this challenge, Machine Learning Operations (MLOps) have emerged as a potential recipe for standardizing ML solutions in deployment. Although MLOps demonstrated great success in streamlining ML processes, thoroughly defining the specifications of robust MLOps approaches remains of great interest to researchers and practitioners. In this paper, we provide a comprehensive overview of the trustworthiness property of MLOps systems. Specifically, we highlight technical practices to achieve robust MLOps systems. In addition, we survey the existing research approaches that address the robustness aspects of ML systems in production. We also review the tools and software available to build MLOps systems and summarize their support to handle the robustness aspects. Finally, we present the open challenges and propose possible future directions and opportunities within this emerging field. The aim of this paper is to provide researchers and practitioners working on practical AI applications with a comprehensive view to adopt robust ML solutions in production environments.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Proceedings of IEEE Conference on Computer Vision and Pattern Recognition
(1993) · doi:10.1109/cvpr.1993 · Cited by 17,667
Chandola, V., Banerjee, A., & Kumar, V. (2009) · ACM Computing Surveys · doi:10.1145/1541880.1541882 · Cited by 11,687 · cited 1×
“However, since building robust solutions requires a systematic way of selecting the splits, random subsampling would not necessarily guarantee the creation of robust splits.”
Machine learning: Trends, perspectives, and prospects
Jordan, M. I., & Mitchell, T. M. (2015) · Science · doi:10.1126/science.aaa8415 · Cited by 9,955 · cited 1×
“Machine learning (ML) , the heart of AI, empowers applications to learn and adapt automatically, paving the way for intelligent and autonomous operation [78].”
Towards Evaluating the Robustness of Neural Networks
Carlini, N., & Wagner, D. (2017) · 2017 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp.2017.49 · Cited by 8,859 · cited 1×
“…of seven fundamental requirements for trustworthy AI systems: (1) human agency and oversight, (2) technical robustness and safety, (3) privacy and data governance, (4) transparency, (5) diversity, non-discrimination, and fairness, (6) environmental and societal well-being, and (7) accountability.”
Weighted kappa: nominal scale agreement provision for scaled disagreement or partial credit
Cohen, J. (1968) · Psychological Bulletin · doi:10.1037/h0026256 · Cited by 8,736 · cited 1×
“The EU requirements for trustworthy AI with different levels of abstraction are summarized in Figure 3.”
Larian Studios Dev Team (2024) · doi:10.1145/3292500.3330701 · Cited by 8,618 · cited 1×
“As discussed in Section 3.2, DataOps involves performing several operations, including data cleaning, ensuring the robustness of the entire ML system to data corruptions such as distributional”
Influential(2012) · Cited by 8,009
The global landscape of AI ethics guidelines
Jobin, A., Ienca, M., & Vayena, E. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0088-2 · Cited by 6,220 · cited 1×
“During the last decades, ML research has been highly dependent on quantitative metrics such as accuracy and loss measures to identify system performance and guide the selection of the ML approach [76].”
Machine learning: Algorithms, real-world applications and research directions
Sarker, I. H. (2021) · SN Computer Science · doi:10.1007/s42979-021-00592-x · Cited by 5,583 · cited 1×
“Next, machine learning models are developed and trained by selecting appropriate algorithms, tuning hyperparameters, and iterating to improve performance [136].”
A few useful things to know about machine learning
Domingos, P. (2012) · Communications of the ACM · doi:10.1145/2347736.2347755 · Cited by 3,326 · cited 1×
“Therefore, most of the effort to build an ML solution is devoted to data cleaning to improve data quality [43].”
Research works printed but not linked: 39
- Hazrat Ali, Christer Grönlund, and Zubair Shah. 2023. Leveraging GANs for data scarcity of COVID-19: Beyond the hype. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 659–667.
- Davide Anguita, Luca Ghelardoni, Alessandro Ghio, Luca Oneto, and Sandro Ridella. 2012. The ‘K’ in K-fold cross validation. In 20th European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning (ESANN). i6doc. com publ, 441–446.
- Firas Bayram, Bestoun S. Ahmed, Erik Hallin, and Anton Engman. 2022. A drift handling approach for self-adaptive ML software in scalable industrial processes. In 2022 37th IEEE/ACM International Conference on Automated Software Engineering (ASE). IEEE.
- Eric Breck, Neoklis Polyzotis, Sudip Roy, Steven Whang, and Martin Zinkevich. 2019. Data validation for machine learning. In MLSys.
- Mandepudi Nobel Chowdary, Bussa Sankeerth, Chennupati Kumar Chowdary, and Manu Gupta. 2022. Accelerating the machine learning model deployment using MLOps. In Journal of Physics: Conference Series, Vol. 2327. IOP Publishing, 012027.
- Abdur Chowdhury and Joshua Alspector. 2003. Data duplication: An imbalance problem?. In ICML’2003 Workshop on Learning from Imbalanced Data Sets (II), Washington, DC. Citeseer.
- European Commission, Content Directorate-General for Communications Networks, and Technology. 2019. Ethics Guidelines for Trustworthy AI. Publications Office.
- Lisa Ehrlinger, Thomas Grubinger, Bence Varga, Mario Pichler, Thomas Natschläger, and Jürgen Zeindl. 2018. Treating missing data in industrial data analytics. In 2018 Thirteenth International Conference on Digital Information Management (ICDIM’18). IEEE, 148–155.
- Xinjian Guo, Yilong Yin, Cailing Dong, Gongping Yang, and Guangtong Zhou. 2008. On the class imbalance problem. In 2008 Fourth International Conference on Natural Computation, Vol. 4. IEEE, 192–201.
- Fabian Hinder, André Artelt, and Barbara Hammer. 2020. Towards non-parametric drift detection via dynamic adapting window independence drift detection (DAWIDD). In International Conference on Machine Learning. PMLR, 4249–4259.
- Hossein Hosseini, Baicen Xiao, and Radha Poovendran. 2017. Google’s Cloud Vision API is not robust to noise. In 2017 16th IEEE International Conference on Machine Learning and Applications (ICMLA’17). IEEE, 101–105.
- Jez Humble and David Farley. 2010. Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation. Pearson Education.
- Davinder Kaur, Suleyman Uslu, and Arjan Durresi. 2020. Requirements for trustworthy artificial intelligence–A review. In International Conference on Network-Based Information Systems. Springer, 105–115.
- Abhishek Kumar, Tristan Braud, Sasu Tarkoma, and Pan Hui. 2020. Trustworthy AI in the age of pervasive computing and big data. In 2020 IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops’20). IEEE, 1–6.
- Robyn Larracy, Angkoon Phinyomark, and Erik Scheme. 2021. Machine learning model validation for early stage studies with small sample sizes. In 2021 43rd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC’21). IEEE, 2314–2319.
- 10.1109/WAIN52551.2021.00024 · DOI · Paper page
- Ankur Mallick, Kevin Hsieh, Behnaz Arzani, and Gauri Joshi. 2022. Matchmaker: Data drift mitigation in machine learning for large-scale systems. Proceedings of Machine Learning and Systems 4 (2022), 77–94.
- Jose Mathew, Meghana Negi, Rutvik Vijjali, and Jairaj Sathyanarayana. 2021. DeFraudNet: An end-to-end weak supervision framework to detect fraud in online food delivery. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases. Springer, 85–99.
- Beatriz M. A. Matsui and Denise H. Goya. 2022. MLOps: A guide to its adoption in the context of responsible AI. In 2022 IEEE/ACM 1st International Workshop on Software Engineering for Responsible Artificial Intelligence (SE4RAI’22). IEEE, 45–49.
- Richard Hugh Moulton, Herna L. Viktor, Nathalie Japkowicz, and João Gama. 2018. Clustering in the presence of concept drift. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases. Springer, 339–355.
- Xiaolei Qian and Gio Wiederhold. 1986. Knowledge-based integrity constraint validation. In VLDB, Vol. 86. Citeseer, 25–28.
- 10.5555/1462129 · doi:10.5555/1462129
- Emmanuel Raj. 2021. Engineering MLOps: Rapidly Build, Test, and Manage Production-ready Machine Learning Life Cycles at Scale. Packt Publishing Ltd.
- Mengye Ren, Wenyuan Zeng, Bin Yang, and Raquel Urtasun. 2018. Learning to reweight examples for robust deep learning. In International Conference on Machine Learning. PMLR, 4334–4343.
- Andras Rozsa, Manuel Günther, and Terrance E. Boult. 2016. Are accuracy and robustness correlated. In 2016 15th IEEE International Conference on Machine Learning and Applications (ICMLA’16). IEEE, 227–232.
- Sebastian Schelter Felix Biessmann Tim Januschowski David Salinas Stephan Seufert and Gyuri Szarvas. 2018. On challenges in machine learning model management. IEEE Data Eng. Bull. 41 (2018) 5–15.
- Shrey Shrivastava, Dhaval Patel, Nianjun Zhou, Arun Iyengar, and Anuradha Bhamidipaty. 2020. DQLearn: A toolkit for structured data quality learning. In 2020 IEEE International Conference on Big Data (Big Data’20). IEEE, 1644–1653.
- Lucas Cardoso Silva, Fernando Rezende Zagatti, Bruno Silva Sette, Lucas Nildaimon dos Santos Silva, Daniel Lucrédio, Diego Furtado Silva, and Helena de Medeiros Caseli. 2020. Benchmarking machine learning solutions in production. In 2020 19th IEEE International Conference on Machine Learning and Applications (ICMLA’20). IEEE, 626–633.
- Masahiro Sugimoto, Masahiro Takada, and Masakazu Toi. 2013. Comparison of robustness against missing values of alternative decision tree and multiple logistic regression for predicting clinical data in primary breast cancer. In 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC’13). IEEE, 3054–3057.
- Georgios Symeonidis, Evangelos Nerantzis, Apostolos Kazakis, and George A. Papakostas. 2022. MLOps-definitions, tools and challenges. In 2022 IEEE 12th Annual Computing and Communication Workshop and Conference (CCWC’22). IEEE, 0453–0460.
- Sunil Thulasidasan, Sushil Thapa, Sayera Dhaubhadel, Gopinath Chennupati, Tanmoy Bhattacharya, and Jeff Bilmes. 2021. An effective baseline for robustness to distributional shift. In 2021 20th IEEE International Conference on Machine Learning and Applications (ICMLA’21). IEEE, 278–285.
- Mark Treveil, Nicolas Omont, Clément Stenac, Kenji Lefevre, Du Phan, Joachim Zentici, Adrien Lavoillotte, Makoto Miyazaki, and Lynn Heidmann. 2020. Introducing MLOps. O’Reilly Media.
- Manasi Vartak, Pablo Ortiz, Kathryn Siegel, Harihar Subramanyam, Samuel Madden, and Matei Zaharia. 2015. Supporting fast iteration in model building. In NIPS Workshop LearningSys. 1–6.
- Robert Woitsch, Wilfrid Utz, Anna Sumereder, Bernhard Dieber, Benjamin Breiling, Laura Crompton, Michael Funk, Karin Bruckmüller, and Stefan Schumann. 2021. Collaborative model-based process assessment for trustworthy AI in robotic platforms. In International Conference on Society 5.0. Springer, 163–174.
- Yinjun Wu, Edgar Dobriban, and Susan Davidson. 2020. DeltaGrad: Rapid retraining of machine learning models. In International Conference on Machine Learning. PMLR, 10355–10366.
- Limin Yang, Wenbo Guo, Qingying Hao, Arridhana Ciptadi, Ali Ahmadzadeh, Xinyu Xing, and Gang Wang. 2021. CADE: Detecting and explaining concept drift samples for security applications. In 30th USENIX Security Symposium (USENIX Security’21). 2327–2344.
- Matei Zaharia, Andrew Chen, Aaron Davidson, Ali Ghodsi, Sue Ann Hong, Andy Konwinski, Siddharth Murching, Tomas Nykodym, Paul Ogilvie, Mani Parkhe, et al. 2018. Accelerating the machine learning lifecycle with MLflow. IEEE Data Eng. Bull. 41, 4 (2018), 39–45.
- Friedemann Zenke, Ben Poole, and Surya Ganguli. 2017. Continual learning through synaptic intelligence. In International Conference on Machine Learning. PMLR, 3987–3995.
- Yizhen Zhao. 2021. Machine Learning in Production: A Literature Review. Universiteit van Amsterdam Technical Report.
Cited by
Data Quality in the Age of AI: A Review of Governance, Ethics, and the FAIR Principles
Guillen-Aguinaga, M., Aguinaga-Ontoso, E., Guillen-Aguinaga, L., Guillen-Grima, F., & Aguinaga-Ontoso, I. (2025) · Data · doi:10.3390/data10120201 · Cited by 37
“…ongoing validation to ensure that training data remain representative of current conditions [18,63,69,181–183, 205 Recent adaptations of the FAIR framework emphasize AI readiness and the need for rigorous attention to provenance, bias detection, and ethical sourcing [25,26,208,236–240].”
Boltaboyeva, A., Baigarayeva, Z., Иманбек, Б., Ozhikenov, К., Getahun, A. J., Aidarova, T., & Karymsakova, N. (2025) · Applied Sciences · doi:10.3390/app15126840 · Cited by 23
Large Language Models for Constructing and Optimizing Machine Learning Workflows: A Survey
Gu, Y., You, H., Cao, J., Yu, M., Fan, H., & Qian, S. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3773084 · Cited by 63
Deep learning for sustainable development across climate, energy, agriculture and urban systems
Sharma, H., & Kaur, S. (2025) · Discover Sustainability · doi:10.1007/s43621-025-02186-6 · Cited by 14
“For example, combining aerial drone footage with soil sensor data enhances agricultural monitoring, while fusing satellite data with policy documents or disaster reports supports contextualized emergency response [10, 18].”
InfluentialSu, Z., Zhang, L., Chen, K., Zhi, L., Huan, X., Chen, F., Wang, X., & Wang, J. (2025) · Gondwana Research · doi:10.1016/j.gr.2025.11.021 · Cited by 5
Metadata
- Authors
- Bayram, F., & Ahmed, B. S.
- Affiliations
- Karlstad University (SE)
- Venue
- ACM Computing Surveys
- Published
- 2025-01-22
- Volume
- 57
- Issue
- 5
- Pages
- 1-35
- ISSN
- 0360-0300 (ISSN-L)
- 1557-7341
- OpenAlex
- W4405547182
- Cited by
- 48
- References
- 174
- Citations by year
- 2026: 31 · 2025: 16
- FWCI
- 7.05
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- ACM Computing Surveys (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion, cc-by)
- Digitala vetenskapliga arkivet (Diva) (Karlstad University) (submittedVersion, other-oa)
- arXiv (Cornell University) (cc-by)
- Funders
- Knowledge Foundation of Sweden (KKS) through the Synergy Project AIDA - A Holistic AI-driven Networking and Processing Framework for Industrial IoT (Rek:20200067)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Computer Networks and Communications › Software System Performance and Reliability
- Keywords
- MLOps, machine learning robustness, trustworthy AI, machine learning in production
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Towards Trustworthy Machine Learning in Production: An Overview of the Robustness in MLOps Approach. https://app.cochat.ai/papers/10.1145/3708497
- The paper
- Bayram, F., & Ahmed, B. S. (2025). Towards Trustworthy Machine Learning in Production: An Overview of the Robustness in MLOps Approach. ACM Computing Surveys. https://doi.org/10.1145/3708497
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.