Towards Trustworthy Machine Learning in Production: An Overview of the Robustness in MLOps Approach
Open accessThe DOI matches its Crossref record; 178 of the 178 references it cites were found in a registry or an index.
Abstract
Artificial intelligence (AI), and especially its sub-field of Machine Learning (ML), are impacting the daily lives of everyone with their ubiquitous applications. In recent years, AI researchers and practitioners have introduced principles and guidelines to build systems that make reliable and trustworthy decisions. From a practical perspective, conventional ML systems process historical data to extract the features that are consequently used to train ML models that perform the desired task. However, in practice, a fundamental challenge arises when the system needs to be operationalized and deployed to evolve and operate in real-life environments continuously. To address this challenge, Machine Learning Operations (MLOps) have emerged as a potential recipe for standardizing ML solutions in deployment. Although MLOps demonstrated great success in streamlining ML processes, thoroughly defining the specifications of robust MLOps approaches remains of great interest to researchers and practitioners. In this paper, we provide a comprehensive overview of the trustworthiness property of MLOps systems. Specifically, we highlight technical practices to achieve robust MLOps systems. In addition, we survey the existing research approaches that address the robustness aspects of ML systems in production. We also review the tools and software available to build MLOps systems and summarize their support to handle the robustness aspects. Finally, we present the open challenges and propose possible future directions and opportunities within this emerging field. The aim of this paper is to provide researchers and practitioners working on practical AI applications with a comprehensive view to adopt robust ML solutions in production environments.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Proceedings of IEEE Conference on Computer Vision and Pattern Recognition
(1993) · doi:10.1109/cvpr.1993 · Cited by 17,667
Chandola, V., Banerjee, A., & Kumar, V. (2009) · ACM Computing Surveys · doi:10.1145/1541880.1541882 · Cited by 11,687 · cited 1×
“However, since building robust solutions requires a systematic way of selecting the splits, random subsampling would not necessarily guarantee the creation of robust splits.”
Machine learning: Trends, perspectives, and prospects
Jordan, M. I., & Mitchell, T. M. (2015) · Science · doi:10.1126/science.aaa8415 · Cited by 9,955 · cited 1×
“Machine learning (ML) , the heart of AI, empowers applications to learn and adapt automatically, paving the way for intelligent and autonomous operation [78].”
Towards Evaluating the Robustness of Neural Networks
Carlini, N., & Wagner, D. (2017) · 2017 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp.2017.49 · Cited by 8,859 · cited 1×
“…of seven fundamental requirements for trustworthy AI systems: (1) human agency and oversight, (2) technical robustness and safety, (3) privacy and data governance, (4) transparency, (5) diversity, non-discrimination, and fairness, (6) environmental and societal well-being, and (7) accountability.”
Weighted kappa: nominal scale agreement provision for scaled disagreement or partial credit
Cohen, J. (1968) · Psychological Bulletin · doi:10.1037/h0026256 · Cited by 8,736 · cited 1×
“The EU requirements for trustworthy AI with different levels of abstraction are summarized in Figure 3.”
Larian Studios Dev Team (2024) · doi:10.1145/3292500.3330701 · Cited by 8,618 · cited 1×
“As discussed in Section 3.2, DataOps involves performing several operations, including data cleaning, ensuring the robustness of the entire ML system to data corruptions such as distributional”
Influential(2012) · Cited by 8,009
The global landscape of AI ethics guidelines
Jobin, A., Ienca, M., & Vayena, E. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0088-2 · Cited by 6,220 · cited 1×
“During the last decades, ML research has been highly dependent on quantitative metrics such as accuracy and loss measures to identify system performance and guide the selection of the ML approach [76].”
Machine learning: Algorithms, real-world applications and research directions
Sarker, I. H. (2021) · SN Computer Science · doi:10.1007/s42979-021-00592-x · Cited by 5,583 · cited 1×
“Next, machine learning models are developed and trained by selecting appropriate algorithms, tuning hyperparameters, and iterating to improve performance [136].”
A few useful things to know about machine learning
Domingos, P. (2012) · Communications of the ACM · doi:10.1145/2347736.2347755 · Cited by 3,326 · cited 1×
“Therefore, most of the effort to build an ML solution is devoted to data cleaning to improve data quality [43].”
An Overview of Overfitting and its Solutions
Ying, X. (2019) · Journal of Physics: Conference Series · doi:10.1088/1742-6596/1168/2/022022 · Cited by 2,352 · cited 1×
“DataOps is a new term 121:11 coined collectively by data engineers, data scientists, and data analysts [118].”
Machine learning algorithm validation with a limited sample size
Vabalas, A., Gowen, E., Poliakoff, E., & Casson, A. J. (2019) · PLOS ONE · doi:10.1371/journal.pone.0224365 · Cited by 1,901 · cited 1×
“ML validation is a diagnostic step that precedes the deployment of the designed solution; it aims to verify the performance under conditions different from those used when building the solution [166].”
A survey on federated learning
Zhang, C., Xie, Y., Bai, H., Yu, B., Li, W., & Gao, Y. (2021) · Knowledge-Based Systems · doi:10.1016/j.knosys.2021.106775 · Cited by 1,899 · cited 1×
“— Heterogeneous Data Sources and Generalization: Distributed systems, such as federated learning, involve training models across multiple devices or institutions with varying datasets that may differ significantly in distribution, quality, and size [193].”
Classification in the Presence of Label Noise: A Survey
Frenay, B., & Verleysen, M. (2014) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2013.2292894 · Cited by 1,777 · cited 1×
“In real-world scenarios, missing values can appear for many reasons, such as sensor failures in industrial problems or incomplete laboratory test results in clinical problems [29].”
Learning under Concept Drift: A Review
Lu, J., Liu, A., Dong, F., Gu, F., Gama, J., & Zhang, G. (2018) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2018.2876857 · Cited by 1,603 · cited 1×
“There are many alternatives to update the model, such as performing re-training, adaptation, or replacement of the model [127].”
Feurer, M., & Hutter, F. (2019) · The Springer Series on Challenges in Machine Learning · doi:10.1007/978-3-030-05318-5_1 · Cited by 1,414
Auto-WEKA: combined selection and hyperparameter optimization of classification algorithms
Thornton, C., Hutter, F., Hoos, H. H., & Leyton-Brown, K. (2013) · Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/2487575.2487629 · Cited by 1,359 · cited 1×
“An update signal is triggered to refresh the learning system to maintain the robustness of the ML model’s performance and keep them up to date with the newly arriving data.”
Techniques for interpretable machine learning
Du, M., Liu, N., & Hu, X. (2019) · Communications of the ACM · doi:10.1145/3359786 · Cited by 1,268 · cited 1×
“Traditionally, the development pipeline of ML systems involves the processing of historical data, which represents the raw input, to extract the features to train an ML model(s) that would perform the desired task offline to produce the output [45].”
Techniques for Interpretable Machine Learning
Du, M., Liu, N., & Hu, X. (2018) · arXiv · doi:10.48550/arxiv.1808.00033 · Cited by 1,261
Machine learning on big data: Opportunities and challenges
Zhou, L., Pan, S., Wang, J., & Vasilakos, A. V. (2017) · Neurocomputing · doi:10.1016/j.neucom.2017.01.026 · Cited by 1,228 · cited 1×
“In the literature, many techniques to perform cross-validation exist.”
Sarker, I. H. (2022) · SN Computer Science · doi:10.1007/s42979-022-01043-x · Cited by 1,208 · cited 1×
“With the increasing difficulty in managing data-driven applications for monitoring and prediction, the deployment of artificial intelligence (AI) systems has played a vital role in real-world applications due to fast-paced advances and innovation in the field [137].”
Generalizing to unseen domains: A survey on domain generalization
Wang, J., Lan, C., Liu, C., Ouyang, Y., Qin, T., Lu, W., Chen, Y., Zeng, W., & Yu, P. (2022) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2022.3178128 · Cited by 1,046 · cited 1×
“Data are pivotal for most of the phases of MLOps pipelines, as they are used to understand the characteristics of our problem and interpret the event episodes of our observations, in addition to practical use to train and validate the ML model.”
Human-in-the-loop machine learning: a state of the art
Mosqueira-Rey, E., Hernández-Pereira, E., Alonso-Ríos, D., Bobes-Bascarán, J., & Fernández-Leal, Á. (2023) · Artificial Intelligence Review · doi:10.1007/s10462-022-10246-w · Cited by 1,025 · cited 1×
“Tiny-MLOps is able to orchestrate the entire life cycle of the MLOps system that executes a specific ML-based task, but faces challenges related to long deployment times for new ML models.”
Machine Learning for Internet of Things Data Analysis: A Survey
Mahdavinejad, M. S., Rezvan, M., Barekatain, M., Adibi, P., Barnaghi, P. M., & Sheth, A. (2017) · Surrey Research Insight Open Access (The University of Surrey) · Cited by 1,008
An architecture-based approach to self-adaptive software
Oreizy, P., Gorlick, M. M., Taylor, R. N., Heimhigner, D., Johnson, G., Medvidovic, N., Quilici, A., Rosenblum, D. S., & Wolf, A. L. (1999) · IEEE Intelligent Systems · doi:10.1109/5254.769885 · Cited by 988 · cited 1×
“The dynamicity of real-world applications requires adaptation in the architectural design of solution systems [120].”
Berrar, D. (2019) · Encyclopedia of Bioinformatics and Computational Biology · doi:10.1016/b978-0-12-809633-8.20349-x · Cited by 958
Incremental Learning of Concept Drift in Nonstationary Environments
Elwell, R., & Polikar, R. (2011) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2011.2160459 · Cited by 931 · cited 1×
“There are many alternatives to update the model, such as performing re-training, adaptation, or replacement of the model [127].”
Machine learning for internet of things data analysis: a survey
Mahdavinejad, M. S., Rezvan, M., Barekatain, M., Adibi, P., Barnaghi, P., & Sheth, A. P. (2018) · Digital Communications and Networks · doi:10.1016/j.dcan.2017.10.002 · Cited by 921 · cited 1×
“ML solutions are mainly based on data manipulation to infer patterns in the system [105].”
Towards a theory of biological robustness
Kitano, H. (2007) · Molecular Systems Biology · doi:10.1038/msb4100179 · Cited by 803 · cited 1×
“Therefore, researchers from different fields have proposed a set of requirements to provide a robust formulation of their problems [86, 101].”
Snorkel: rapid training data creation with weak supervision
Ratner, A., Bach, S. H., Ehrenberg, H., Fries, J., Wu, S., & Ré, C. (2017) · Proceedings of the VLDB Endowment · doi:10.14778/3157794.3157797 · Cited by 777 · cited 3×
“Moreover, the versioning mechanism should clearly define what constitutes an iteration, identifying variations in ML artifacts to uniquely distinguish each iteration in the MLOps lifecycle.”
Machine Learning Operations (MLOps): Overview, Definition, and Architecture
Kreuzberger, D., Kühl, N., & Hirschl, S. (2023) · IEEE Access · doi:10.1109/access.2023.3262138 · Cited by 766 · cited 1×
“Research has also explored the effectiveness of GANs in the energy sector for power consumption prediction and in defect detection [183].”
Representations of quasi-Newton matrices and their use in limited memory methods
Byrd, R. H., Nocedal, J., & Schnabel, R. B. (1994) · Mathematical Programming · doi:10.1007/bf01582063 · Cited by 744 · cited 1×
“Specifically, we survey the methods that are concerned with the robustness aspects that we presented in Section 3.”
Bradley, J. V. (1978) · British Journal of Mathematical and Statistical Psychology · doi:10.1111/j.2044-8317.1978.tb00581.x · Cited by 739
Direct importance estimation with model selection and its application to covariate shift adaptation
Sugiyama, M., Nakajima, S., Kashima, H., von Bünau, P., & Kawanabe, M. (2009) · Publikationsdatenbank der Fraunhofer-Gesellschaft (Fraunhofer-Gesellschaft) · Cited by 719
Trustworthy Artificial Intelligence: A Review
Kaur, D., Uslu, S., Rittichier, K. J., & Durresi, A. (2023) · ACM Computing Surveys · doi:10.1145/3491209 · Cited by 709
Dataset Shift in Machine Learning
Adams, N. (2010) · Journal of the Royal Statistical Society Series A: Statistics in Society · doi:10.1111/j.1467-985x.2009.00624_10.x · Cited by 694
A Survey on Machine Learning for Data Fusion
Meng, T., Jing, X., Yan, Z., & Pedrycz, W. (2020) · Information Fusion · doi:10.1016/j.inffus.2019.12.001 · Cited by 677 · cited 1×
“Data fusion is the most widely used approach to correct data duplication [114].”
Machine learning and data mining in manufacturing
Dogan, A., & Birant, D. (2021) · Expert Systems with Applications · doi:10.1016/j.eswa.2020.114060 · Cited by 677 · cited 1×
“Typically, the environment and its conditions are dynamic and, as the systems evolve, the MLOps systems should perform a set of operations to keep the ML models up-to-date [42].”
Botchkarev, A. (2019) · Interdisciplinary Journal of Information, Knowledge, and Management · doi:10.28945/4184 · Cited by 616 · cited 1×
“The EU requirements for trustworthy AI with different levels of abstraction are summarized in Figure 3.”
Trustworthy AI: From Principles to Practices
Li, B., Qi, P., Liu, B., Di, S., Liu, J., Pei, J., Yi, J., & Zhou, B. (2023) · ACM Computing Surveys · doi:10.1145/3555803 · Cited by 612 · cited 3×
“These AI systems, equipped with various toolkits, offer a wide range of solutions that can be adopted to help decision makers take rapid action in various applications [3].”
Zhang, L., Wang, X., Yang, D., Sanford, T., Harmon, S., Turkbey, B., Wood, B. J., Roth, H., Myronenko, A., Xu, D., et al. (2020) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2020.2973595 · Cited by 542 · cited 1×
“…processing Prone to false positives [147] VAE-based detection of changes without ground-truth labels Evaluated using use-cases not fully representing real-word scenarios [14] Fully automated and scalable Supports limited ML models [33] Can handle complex and high-dimensional features Only focused…”
Snorkel: rapid training data creation with weak supervision
Ratner, A., Bach, S. H., Ehrenberg, H., Fries, J., Wu, S., & Ré, C. (2020) · The VLDB Journal · doi:10.1007/s00778-019-00552-1 · Cited by 510
Data Cleaning: Overview and Emerging Challenges
Chu, X., Ilyas, I. F., Krishnan, S., & Wang, J. (2016) · Proceedings of the 2016 International Conference on Management of Data · doi:10.1145/2882903.2912574 · Cited by 488 · cited 1×
“The hyperparameter settings of the model are considered an aspect of model robustness [46] and will be further investigated in a separate section.”
From concept drift to model degradation: An overview on performance-aware drift detectors
Bayram, F., Ahmed, B. S., & Kassler, A. (2022) · Knowledge-Based Systems · doi:10.1016/j.knosys.2022.108632 · Cited by 407
Self-citationBotchkarev, A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.03006 · Cited by 387 · cited 1×
“The EU requirements for trustworthy AI with different levels of abstraction are summarized in Figure 3.”
Usuga Cadavid, J. P., Lamouri, S., Grabot, B., Pellerin, R., & Fortin, A. (2020) · Journal of Intelligent Manufacturing · doi:10.1007/s10845-019-01531-7 · Cited by 387 · cited 1×
“Our main focus is the robustness property, as it is crucial for guaranteeing the reliability and performance of models in real-world scenarios where data variability is common, ensuring the effectiveness of AI solutions in production environments [152].”
Improving Generalization Performance by Switching from Adam to SGD
Keskar, N. S., & Socher, R. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1712.07628 · Cited by 373 · cited 1×
“The pros and cons of the various approaches are outlined in Table 4 to provide a comprehensive summary.”
A review of automatic selection methods for machine learning algorithms and hyper-parameter values
Luo, G. (2016) · Network Modeling Analysis in Health Informatics and Bioinformatics · doi:10.1007/s13721-016-0125-6 · Cited by 357 · cited 1×
“The challenge of the hyperparameter tuning methods of the ML model in production is the search space, which is very large for most learning algorithms [99], especially complex algorithms such as neural networks [52].”
TFX: A TensorFlow-Based Production-Scale Machine Learning Platform
Baylor, D., Breck, E., Cheng, H.-T., Fiedel, N., Foo, C. Y., Haque, Z., Haykal, S., Ispir, M., Jain, V., Koc, L., et al. (2017) · Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/3097983.3098021 · Cited by 350 · cited 1×
“In addition, the platform also supports anomaly detection using statistics from historical data quality metrics, but real-time capabilities should be enhanced.”
A Closer Look at Accuracy vs. RobustnessPreprint
Yang, Y.-Y., Rashtchian, C., Change Institutions to: University of Waterloo, Salakhutdinov, R., & Chaudhuri, K. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.02460 · Cited by 330
Data Quality: The Accuracy Dimension
Olson, J. E. (2003) · Cited by 327
Robustness May Be at Odds with Accuracy
Tsipras, D., Santurkar, S., Engstrom, L., Turner, A., & Mądry, A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.12152 · Cited by 327
The ML test score: A rubric for ML production readiness and technical debt reduction
Breck, E., Cai, S., Nielsen, E., Salib, M., & Sculley, D. (2017) · 2017 IEEE International Conference on Big Data (Big Data) · doi:10.1109/bigdata.2017.8258038 · Cited by 310 · cited 2×
“Furthermore, the article discussed approaches to mitigate AI risks by involving users and society to enrich AI systems.”
Magician’s Corner: 9. Performance Metrics for Machine Learning Models
Erickson, B. J., & Kitamura, F. (2021) · Radiology: Artificial Intelligence · doi:10.1148/ryai.2021200126 · Cited by 306 · cited 1×
“The EU requirements for trustworthy AI with different levels of abstraction are summarized in Figure 3.”
A Systematic Review on Data Scarcity Problem in Deep Learning: Solution and Applications
Bansal, M. A., Sharma, D. R., & Kathuria, D. M. (2022) · ACM Computing Surveys · doi:10.1145/3502287 · Cited by 306 · cited 1×
“Moreover, the versioning mechanism should clearly define what constitutes an iteration, identifying variations in ML artifacts to uniquely distinguish each iteration in the MLOps lifecycle.”
Learning from streaming data with concept drift and imbalance: an overview
Hoens, T. R., Polikar, R., & Chawla, N. V. (2012) · Progress in Artificial Intelligence · doi:10.1007/s13748-011-0008-0 · Cited by 301 · cited 1×
“The data used to train ML algorithms is assumed to be sampled from a distribution defined by an underlying generating function [66].”
The adverse effects of code duplication in machine learning models of code
Allamanis, M. (2019) · Proceedings of the 2019 ACM SIGPLAN International Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software · doi:10.1145/3359591.3359735 · Cited by 284 · cited 1×
“Similarly to missing values, the duplication of data samples has a negative implication in learning, as it can skew the input distribution [35], resulting in duplicate bias [7].”
Machine Learning in Non-Stationary Environments: Introduction to Covariate Shift Adaptation
Sugiyama, M., & Kawanabe, M. (2012) · Cited by 263
Chen, L. (2021) · Cognitive Intelligence and Robotics · doi:10.1007/978-981-16-2233-5_11 · Cited by 263
Trustworthy ai: A computational perspective
Liu, H., Wang, Y., Fan, W., Liu, X., Li, Y., Jain, S., Liu, Y., Jain, A., & Tang, J. (2023) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3546872 · Cited by 261
Wu, H., Zhang, Z., Guan, C., Wolter, K., & Xu, M. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.2996784 · Cited by 260
Raschka, S., & Mirjalili, V. (2017) · Cited by 255
Ho, S. Y., Phua, K., Wong, L., & Bin Goh, W. W. (2020) · Patterns · doi:10.1016/j.patter.2020.100129 · Cited by 247 · cited 2×
“…new concepts [107] Scalable, adaptable, and flexible for large-scale systems Cannot handle unseen drifts leading to accuracy drops Generalization [65] Handles high-dimensional data Limited to specific tasks [194] Efficient in unseen domains Handles supervised tasks only [61] Reduces…”
Automating large-scale data quality verification
Schelter, S., Lange, D., Schmidt, P., Celikel, M., Biessmann, F., & Grafberger, A. (2018) · Proceedings of the VLDB Endowment · doi:10.14778/3229863.3229867 · Cited by 240 · cited 2×
“[80] Handling missing data efficiently Requires some manual efforts [139] Scalable to large datasets Not validated in streaming scenarios [24] Supports single-batch and inter-batch validation System’s implementation influenced by Google’s infrastructure [188] User-friendly platform without the need…”
Concept Drift Detection in Data Stream Mining : A literature review
Agrahari, S., & Singh, A. K. (2022) · Journal of King Saud University - Computer and Information Sciences · doi:10.1016/j.jksuci.2021.11.006 · Cited by 233 · cited 1×
“There are many alternatives to update the model, such as performing re-training, adaptation, or replacement of the model [127].”
Privacy Risks of Securing Machine Learning Models against Adversarial Examples
Song, L., Shokri, R., & Mittal, P. (2019) · Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3319535.3354211 · Cited by 229 · cited 2×
“At the conceptual level, according to EU guidelines, to make an AI system trustworthy, it should align three main components at the top level throughout its evolution; the components are summarized as follows [82]: (1) Lawful : it ensures that the AI system must comply with all applicable laws and regulations, such as…”
A Study on the Relationships of Classifier Performance Metrics
Seliya, N., Khoshgoftaar, T. M., & Van Hulse, J. (2009) · 2009 21st IEEE International Conference on Tools with Artificial Intelligence · doi:10.1109/ictai.2009.25 · Cited by 224 · cited 1×
“The EU requirements for trustworthy AI with different levels of abstraction are summarized in Figure 3.”
Machine Learning in Non-Stationary Environments: Introduction to Covariate Shift Adaptation
Sugiyama, M., & Kawanabe, M. (2012) · The MIT Press eBooks · doi:10.7551/mitpress/9780262017091.001.0001 · Cited by 223
Enhancing robustness of machine learning systems via data transformations
Bhagoji, A. N., Cullina, D., Sitawarin, C., & Mittal, P. (2018) · 2018 52nd Annual Conference on Information Sciences and Systems (CISS) · doi:10.1109/ciss.2018.8362326 · Cited by 217
Ilyas, I. F., & Chu, X. (2019) · ACM eBooks · doi:10.1145/3310205 · Cited by 197
Dynamic integration of classifiers for handling concept drift
Tsymbal, A., Pechenizkiy, M., Cunningham, P., & Puuronen, S. (2008) · Information Fusion · doi:10.1016/j.inffus.2006.11.002 · Cited by 195 · cited 1×
“However, in practice, the model should always be reviewed when there is a change in the data distribution [164].”
Adaptive concept drift detection
Dries, A., & Rückert, U. (2009) · Statistical Analysis and Data Mining The ASA Data Science Journal · doi:10.1002/sam.10054 · Cited by 191 · cited 1×
“In deployment, MLOps systems should exhibit high robustness to distributional shifts; this requires systematic detection and adaptation mechanisms to cope with changes [44].”
Sequential change‐point detection based on direct density‐ratio estimation
Kawahara, Y., & Sugiyama, M. (2012) · Statistical Analysis and Data Mining: The ASA Data Science Journal · doi:10.1002/sam.10124 · Cited by 190 · cited 1×
“Regarding distributional shifts, monitoring is generally performed by determining whether the data distribution of the present interval is significantly different from the past distribution using statistical tests [84].”
Trustworthy AI and robotics: Implications for the AEC industry
Emaminejad, N., & Akhavian, R. (2022) · Automation in Construction · doi:10.1016/j.autcon.2022.104298 · Cited by 174 · cited 1×
“Therefore, trustworthy AI has been introduced in numerous application domains, such as self-driving cars [81], dentistry [103], the architecture, engineering and construction (AEC) industry [50], and education [170].”
Performance Anomaly Detection and Bottleneck Identification
Ibidunmoye, O., Hernández-Rodriguez, F., & Elmroth, E. (2015) · ACM Computing Surveys · doi:10.1145/2791120 · Cited by 173 · cited 1×
“In addition to the negative effect that anomalies have on the performance of predictive systems [72], anomalies can indicate malicious activities or adversarial attacks [74], which can affect the robustness of the overall system.”
Who Needs MLOps: What Data Scientists Seek to Accomplish and How Can MLOps Help?Preprint
Mäkinen, S., Skogström, H., Laaksonen, E., & Mikkonen, T. (2021) · arXiv (Cornell University) · Cited by 169
Effect of label noise in the complexity of classification problems
Garcia, L. P. F., de Carvalho, A. C. P. L. F., & Lorena, A. C. (2015) · Neurocomputing · doi:10.1016/j.neucom.2014.10.085 · Cited by 167 · cited 2×
“The removal of noisy labels is not always feasible, as it is difficult to determine whether the labels are 121:15 indeed noisy [54].”
MAPE-K Formal Templates to Rigorously Design Behaviors for Self-Adaptive Systems
Iglesia, D. G. D. L., & Weyns, D. (2015) · ACM Transactions on Autonomous and Adaptive Systems · doi:10.1145/2724719 · Cited by 167 · cited 1×
“The framework was built based on Monitor-Analyse-Plan-Execute over a shared Knowledge (MAPE-K) architecture for self-adaptation [73].”
Open-environment machine learning
Zhou, Z. (2022) · National Science Review · doi:10.1093/nsr/nwac123 · Cited by 166
Combined Cleaning and Resampling algorithm for multi-class imbalanced data with label noise
Koziarski, M., Woźniak, M., & Krawczyk, B. (2020) · Knowledge-Based Systems · doi:10.1016/j.knosys.2020.106223 · Cited by 165
Sherpa: Robust hyperparameter optimization for machine learning
Hertel, L., Collado, J., Sadowski, P., Ott, J., & Baldi, P. (2020) · SoftwareX · doi:10.1016/j.softx.2020.100591 · Cited by 159
A survey on robustness in railway planning
Lusby, R. M., Larsen, J., & Bull, S. H. (2017) · European Journal of Operational Research · doi:10.1016/j.ejor.2017.07.044 · Cited by 157
Robustness and Explainability of Artificial Intelligence
(2020) · Joint Research Centre (European Commission) · doi:10.2760/57493 · Cited by 155
Elastic Machine Learning Algorithms in Amazon SageMaker
Liberty, E., Karnin, Z. S., Xiang, B., Rouesnel, L., Coşkun, B., Nallapati, R. M., Delgado, J., Sadoughi, A., Astashonok, Y., Das, P., et al. (2020) · ACM SIGMOD International Conference on Management of Data (SIGMOD) · doi:10.1145/3318464.3386126 · Cited by 155 · cited 1×
“From a technological perspective, the most popular public tools that manage the ML life cycle in deployments are [159]: Amazon SageMaker by Amazon Web Service (AWS) [95], Microsoft Azure [146], Vertex AI (formerly known as AI Platform) [20], MLFlow [189], and TensorFlow Extended (TFX) [13].”
Wing, J. M. (2021) · Communications of the ACM · doi:10.1145/3448248 · Cited by 148 · cited 2×
“The aspects and principles are known as AI trustworthiness [175].”
Wagstaff, K. (2012) · arXiv · doi:10.48550/arxiv.1206.4656 · Cited by 148 · cited 1×
“(3) Robust : it ensures that the AI system is technically and socially reliable so that it does not generate harmful risks that AI systems can cause.”
Applying DevOps Practices of Continuous Automation for Machine Learning
Karamitsos, I., Thabit, S., & Apostolopoulos, C. (2020) · Information · doi:10.3390/info11070363 · Cited by 140 · cited 1×
“EU’s requirement for trustworthy AI inspired by [38, 83].”
Data scarcity, robustness and extreme multi-label classification
Babbar, R., & Schölkopf, B. (2019) · Machine Learning · doi:10.1007/s10994-019-05791-5 · Cited by 136 · cited 1×
“On the one hand, using a unique version number allows us to locate and trace the roots of the fault, and on the other hand, it also allows us to reuse a specific state of the system [182].”
Understanding DevOps & bridging the gap from continuous integration to continuous delivery
Virmani, M. (2015) · Fifth International Conference on the Innovative Computing Technology (INTECH 2015) · doi:10.1109/intech.2015.7173368 · Cited by 132 · cited 1×
“EU’s requirement for trustworthy AI inspired by [38, 83].”
Time Series Data Cleaning: A Survey
Wang, X., & Wang, C. (2020) · IEEE Access · doi:10.1109/access.2019.2962152 · Cited by 132 · cited 2×
“Furthermore, the implementation of a well-defined validation mechanism not only leads to more reproducible solutions [83] but also helps mitigate the risks associated with ML models in general, such as security concerns [111], adversarial attacks [148], and risks related to algorithms and data…”
Towards MLOps: A Framework and Maturity Model
John, M. M., Olsson, H. H., & Bosch, J. (2021) · 2021 47th Euromicro Conference on Software Engineering and Advanced Applications (SEAA) · doi:10.1109/seaa53835.2021.00050 · Cited by 130 · cited 1×
“Research has also explored the effectiveness of GANs in the energy sector for power consumption prediction and in defect detection [183].”
Real-time data management on wireless sensor networks: A survey
Diallo, O., Rodrigues, J. J. P. C., & Sène, M. (2011) · Journal of Network and Computer Applications · doi:10.1016/j.jnca.2011.12.006 · Cited by 129 · cited 1×
“Undoubtedly, performing different data operations requires huge computing and storage resources that should typically be performed in real-time, which causes great difficulty for production systems [41].”
Steidl, M., Felderer, M., & Ramler, R. (2023) · Journal of Systems and Software · doi:10.1016/j.jss.2023.111615 · Cited by 126
Sustainable MLOps: Trends and Challenges
Tamburri, D. A. (2020) · 2020 22nd International Symposium on Symbolic and Numeric Algorithms for Scientific Computing (SYNASC) · doi:10.1109/synasc51798.2020.00015 · Cited by 124 · cited 1×
“Research has also explored the effectiveness of GANs in the energy sector for power consumption prediction and in defect detection [183].”
Adversarially robust transfer learningPreprint
Shafahi, A., Saadatpanah, P., Zhu, C., Ghiasi, A., Studer, C., Jacobs, D. W., & Goldstein, T. (2019) · arXiv (Cornell University) · doi:10.3929/ethz-b-000458750 · Cited by 123
Accelerating Human-in-the-loop Machine Learning: Challenges and Opportunities
Xin, D., Ma, L., Liu, J., Macke, S., Song, S., & Parameswaran, A. (2018) · Proceedings of the Second Workshop on Data Management for End-To-End Machine Learning · doi:10.1145/3209889.3209897 · Cited by 120 · cited 1×
“Meanwhile, another detailed guide document has recently been published that surveys the topics relevant to building trustworthy AI systems has recently been published by the International Organization for Standardization (ISO) and the International Electrotechnical Commission (IEC) [75].”
From Ad-Hoc Data Analytics to DataOps
Munappy, A. R., Mattos, D. I., Bosch, J., Olsson, H. H., & Dakkak, A. (2020) · International Conference on Software and System Process · doi:10.1145/3379177.3388909 · Cited by 120 · cited 1×
“DataOps is a new term 121:11 coined collectively by data engineers, data scientists, and data analysts [118].”
Machine learning for science and society
Rudin, C., & Wagstaff, K. L. (2014) · Machine Learning · doi:10.1007/s10994-013-5425-9 · Cited by 114 · cited 1×
“The fundamental challenge arises when the ML solution needs to be operationalized and deployed to function in a real-world environment [134].”
MLOps: A Taxonomy and a Methodology
Testi, M., Ballabio, M., Frontoni, E., Iannello, G., Moccia, S., Soda, P., & Vessio, G. (2022) · IEEE Access · doi:10.1109/access.2022.3181730 · Cited by 114 · cited 2×
“The platform integrates an incremental validation methodology of data quality for evolving datasets.”
Chatila, R., Dignum, V., Fisher, M., Giannotti, F., Morik, K. J., Russell, S., & Yeung, K. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-69128-8_2 · Cited by 113
Demystifying MLOps and Presenting a Recipe for the Selection of Open-Source Tools
Ruf, P., Madan, M., Reich, C., & Ould-Abdeslam, D. (2021) · Applied Sciences · doi:10.3390/app11198861 · Cited by 112 · cited 1×
“MLOps is an extension of DevOps, which is a well-known and mature topic in software engineering, to run ML systems in production [135].”
Recent Advances in Supervised Dimension Reduction: A Survey
Chao, G., Luo, Y., & Ding, W. (2019) · Machine Learning and Knowledge Extraction · doi:10.3390/make1010020 · Cited by 111 · cited 1×
“In particular, how to select the validation datasets and performance metrics to evaluate the performance of the tested solution.”
Generalization in Reinforcement Learning by Soft Data Augmentation
Hansen, N., & Wang, X. (2021) · 2021 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra48506.2021.9561103 · Cited by 111 · cited 1×
“…inference Only applicable to certain tasks [129] Supports multitask learning and a code-free approach Does not have support for model versioning [128] Reduces the cost and difficulty of training ML models Requires large amount of unlabeled data [31] Uses domain knowledge to generate synthetic data”
Understanding and Visualizing Data Iteration in Machine Learning
Hohman, F., Wongsuphasawat, K., Kery, M. B., & Patel, K. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376177 · Cited by 110 · cited 1×
“A robust versioning mechanism must ensure that the different ML artifacts, such as configuration and hyperparameters, data and algorithm, have consistent version numbers for a specific iteration [67].”
Luo, Y., Szolovits, P., Dighe, A. S., & Baron, J. M. (2017) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocx133 · Cited by 108 · cited 1×
“Traditionally, missing values are handled by discarding the tuple that includes incomplete data or replacing missing values with a statistically inferred value, known as imputation [100].”
ModelOps: Cloud-Based Lifecycle Management for Reliable and Trusted AI
Hummer, W., Muthusamy, V., Rausch, T., Dube, P., El Maghraoui, K., Murthi, A., & Oum, P. (2019) · 2019 IEEE International Conference on Cloud Engineering (IC2E) · doi:10.1109/ic2e.2019.00025 · Cited by 105 · cited 1×
“Since the construction of MLOps systems involves the integration of several components into the system infrastructure, it is essential to establish a comprehensive monitoring strategy to cover these components.”
Monitoring and explainability of models in productionPreprint
Klaise, J., Looveren, A. V., Cox, C., Vacanti, G., & Coca, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.06299 · Cited by 105 · cited 1×
“MLOps provides a comprehensive framework for managing the entire lifecycle of ML models, including data acquisition, data preparation, model training, testing, evaluation, deployment, and continuous monitoring.”
Du, X., Xu, H., & Zhu, F. (2021) · Computer-Aided Design · doi:10.1016/j.cad.2021.103013 · Cited by 99
Principles and Practice of Explainable Machine Learning
Belle, V., & Papantonis, I. (2021) · Frontiers in Big Data · doi:10.3389/fdata.2021.688969 · Cited by 99 · cited 1×
“Additionally, the use of explainable AI techniques can improve our understanding of system behavior and observations, facilitating the construction of quantifiable solutions [16].”
Time Series Prediction Method of Industrial Process With Limited Data Based on Transfer Learning
Zhou, X., Zhai, N., Li, S., & Shi, H. (2022) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2022.3191980 · Cited by 98
Robust Optimization and Validation of Echo State Networks for learning chaotic dynamics
Racca, A., & Magri, L. (2021) · Neural Networks · doi:10.1016/j.neunet.2021.05.004 · Cited by 96
Change with Delayed Labeling: When is it Detectable?
Žliobaitė, I. (2010) · IEEE ... International Conference on Data Mining workshops · doi:10.1109/icdmw.2010.49 · Cited by 94 · cited 1×
“It begins with ensuring high-quality, reliable data through the collection, cleaning, and preprocessing of raw data to make it suitable for model training.”
Edge MLOps: An Automation Framework for AIoT Applications
Raj, E., Buffoni, D., Westerlund, M., & Ahola, K. (2021) · IEEE International Conference on Cloud Engineering · doi:10.1109/ic2e52221.2021.00034 · Cited by 93 · cited 4×
“ModelOps should find the balance between the complexity of the concept drift handling solution and the time required to execute the adaptation, as concept drift could have severe consequences for the overall system if not addressed quickly.”
InfluentialDynamic Weighted Majority for Incremental Learning of Imbalanced Data Streams with Concept Drift
Lu, Y., Cheung, Y., & Tang, Y. Y. (2017) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2017/333 · Cited by 92 · cited 1×
“The pros and cons of the various approaches are outlined in Table 4 to provide a comprehensive summary.”
Garg, S., Pundir, P., Rathee, G., Gupta, P. K., Garg, S., & Ahlawat, S. (2021) · 2021 IEEE Fourth International Conference on Artificial Intelligence and Knowledge Engineering (AIKE) · doi:10.1109/aike52691.2021.00010 · Cited by 91
Generalizing AI: Challenges and Opportunities for Plug and Play AI Solutions
Ridhawi, I. A., Otoum, S., Aloqaily, M., & Boukerche, A. (2020) · IEEE Network · doi:10.1109/mnet.011.2000371 · Cited by 88 · cited 1×
“These AI systems, equipped with various toolkits, offer a wide range of solutions that can be adopted to help decision makers take rapid action in various applications [3].”
Facilitating Knowledge Sharing from Domain Experts to Data Scientists for Building NLP Models
Park, S., Wang, A. Y., Kawas, B., Liao, Q. V., Piorkowski, D., & Danilevsky, M. (2021) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3397481.3450637 · Cited by 88
Crankshaw, D., Bailis, P. D., Gonzalez, J. E., Li, H., Zhang, Z., Franklin, M. J. M., Ghodsi, A., & Jordan, M. I. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1409.3809 · Cited by 84 · cited 2×
“Therefore, any corruption in historical data would lead to poor deployment decisions.”
An Overview of Google Cloud Platform Services
Bisong, E. (2019) · Apress eBooks · doi:10.1007/978-1-4842-4470-8_2 · Cited by 79 · cited 1×
“In addition, the platform also supports anomaly detection using statistics from historical data quality metrics, but real-time capabilities should be enhanced.”
On the Impact of Sample Duplication in Machine-Learning-Based Android Malware Detection
Zhao, Y., Li, L., Wang, H., Cai, H., Bissyandé, T. F., Klein, J., & Grundy, J. (2021) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3446905 · Cited by 76 · cited 1×
“Mostly, validation splits should include outliers, missing data, or distribution shifts.”
Assessing Trust Level of a Driverless Car Using Deep Learning
Karmakar, G., Chowdhury, A., Das, R., Kamruzzaman, J., & Islam, S. M. (2021) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2021.3059261 · Cited by 72 · cited 1×
“Therefore, trustworthy AI has been introduced in numerous application domains, such as self-driving cars [81], dentistry [103], the architecture, engineering and construction (AEC) industry [50], and education [170].”
Sarker, I. H. (2022) · Preprints.org · doi:10.20944/preprints202202.0001.v1 · Cited by 72
The machine learning life cycle and the cloud: implications for drug discovery
Spjuth, O., Frid, J., & Hellander, A. (2021) · Expert Opinion on Drug Discovery · doi:10.1080/17460441.2021.1932812 · Cited by 67 · cited 1×
“EU’s requirement for trustworthy AI inspired by [38, 83].”
Towards Trustworthy AI in Dentistry
Ma, J., Schneider, L., Lapuschkin, S., Achtibat, R., Duchrau, M., Krois, J., Schwendicke, F., & Samek, W. (2022) · Journal of Dental Research · doi:10.1177/00220345221106086 · Cited by 66
Domain Generalization in Vision: A SurveyPreprint
Zhou, K., Liu, Z., Qiao, Y., Xiang, T., & Loy, C. C. (2021) · arXiv (Cornell University) · Cited by 65
Alla, S., & Adari, S. K. (2020) · Apress eBooks · doi:10.1007/978-1-4842-6549-9_3 · Cited by 61 · cited 1×
“Moreover, the versioning mechanism should clearly define what constitutes an iteration, identifying variations in ML artifacts to uniquely distinguish each iteration in the MLOps lifecycle.”
Tiny-MLOps: a framework for orchestrating ML applications at the far edge of IoT systems
Antonini, M., Pincheira, M., Vecchio, M., & Antonelli, F. (2022) · IEEE Conference on Evolving and Adaptive Intelligent Systems (EAIS) · doi:10.1109/eais51927.2022.9787703 · Cited by 54 · cited 1×
“In addition to prediction performance, the approach is designed to monitor five metrics of resource consumption: completion time, CPU and GPU usage, memory usage, disk input/output (IO) operations, and network traffic.”
Machine Learning Operations (MLOps): Overview, Definition, and ArchitecturePreprint
Kreuzberger, D., Kühl, N., & Hirschl, S. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2205.02302 · Cited by 51 · cited 1×
“Research has also explored the effectiveness of GANs in the energy sector for power consumption prediction and in defect detection [183].”
Scarcity-GAN: Scarce data augmentation for defect detection via generative adversarial nets
Xu, C., Li, W., Cui, X., Wang, Z., Zheng, F., Zhang, X., & Chen, B. (2023) · Neurocomputing · doi:10.1016/j.neucom.2023.127061 · Cited by 51 · cited 1×
“Research has also explored the effectiveness of GANs in the energy sector for power consumption prediction and in defect detection [183].”
Reflections on Artificial Intelligence for Humanity
Braunschweig, B., & Ghallab, M. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-69128-8 · Cited by 48
A Segment-Based Drift Adaptation Method for Data Streams
Song, Y., Lü, J., Liu, A., Lu, H., & Zhang, G. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3062062 · Cited by 47 · cited 1×
“There are many alternatives to update the model, such as performing re-training, adaptation, or replacement of the model [127].”
Medhat, N., Moussa, S. M., Badr, N. L., & Tolba, M. F. (2020) · IEEE Access · doi:10.1109/access.2020.3039931 · Cited by 45
PrIU: A Provenance-Based Approach for Incrementally Updating Regression Models
Wu, Y., Tannen, V., & Davidson, S. B. (2020) · ACM SIGMOD International Conference on Management of Data (SIGMOD) · doi:10.1145/3318464.3380571 · Cited by 42 · cited 2×
“To compile the latest approaches devoted to developing ML systems that address the robustness property in deployment, we conduct an extensive literature search for existing solutions that handle one or more robustness aspects in production that have been proposed in the research community.”
A Data Quality-Driven View of MLOps
Renggli, C., Rimanić, L., Gürel, N. M., Karlaš, B., Wu, W., & Zhang, C. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.07750 · Cited by 40 · cited 1×
“In such scenarios, additional constraints and limitations should be specified to perform the tasks that characterize the ML solution, as the offline solution may not meet real-world requirements.”
From DevOps to DevDataOps: Data Management in DevOps Processes
Capizzi, A., Distefano, S., & Mazzara, M. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-39306-9_4 · Cited by 39
Karlaš, B., Li, P., Wu, R., Gürel, N. M., Chu, X., Wu, W., & Zhang, C. (2020) · Proceedings of the VLDB Endowment · doi:10.14778/3430915.3430917 · Cited by 39
Aeromagnetic Compensation Algorithm Robust to Outliers of Magnetic Sensor Based on Huber Loss Method
Ge, J., Li, H., Wang, H., Dong, H., Liu, H., Wang, W., Yuan, Z., Zhu, J., & Zhang, H. (2019) · IEEE Sensors Journal · doi:10.1109/jsen.2019.2907398 · Cited by 38
Machine Learning Operations: A Survey on MLOps Tool Support
Hewage, N., & Meedeniya, D. (2022) · arXiv · doi:10.48550/arxiv.2202.10169 · Cited by 38
AIOps: Predictive Analytics & Machine Learning in Operations
Masood, A., & Hashmi, A. (2019) · Apress eBooks · doi:10.1007/978-1-4842-4106-6_7 · Cited by 37
Alla, S., & Adari, S. K. (2020) · Apress eBooks · doi:10.1007/978-1-4842-6549-9 · Cited by 37
CheXstray: Real-time Multi-Modal Data Concordance for Drift Detection in Medical Imaging AIPreprint
Soin, A., Merkow, J., Long, J., Cohen, J., Saligrama, S., Kaiser, S., Borg, S., Tarapov, I., & Lungren, M. P. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2202.02833 · Cited by 37 · cited 1×
“In another article [30], Chatila et al. also discussed the requirements to design trustworthy AI systems.”
Overton: A Data System for Monitoring and Improving Machine-Learned ProductsPreprint
Ré, C., Niu, F., Gudipati, P., & Srisuwananukorn, C. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.05372 · Cited by 36 · cited 2×
“Furthermore, the article discussed approaches to mitigate AI risks by involving users and society to enrich AI systems.”
Sparse Bayesian modelling of underreported count data
Dvorzak, M., & Wagner, H. (2015) · Statistical Modelling · doi:10.1177/1471082x15588398 · Cited by 32 · cited 1×
“The ML artifacts will follow changes over time, so it is fundamental to record the historical versions to obtain robust and tractable MLOps systems.”
The Top 10 Risks of Machine Learning Security
McGraw, G. E., Bonett, R., Shepardson, V., & Figueroa, H. (2020) · Computer · doi:10.1109/mc.2020.2984868 · Cited by 31 · cited 1×
“…trustworthy, it should align three main components at the top level throughout its evolution; the components are summarized as follows [82]: (1) Lawful : it ensures that the AI system must comply with all applicable laws and regulations, such as the General Data Protection Regulation (GDPR) .”
Zeller, M., Waschulzik, T., Schmid, R., & Bahlmann, C. (2024) · AI and Ethics · doi:10.1007/s43681-023-00392-4 · Cited by 31
Versioning for End-to-End Machine Learning Pipelines
van der Weide, T., Papadopoulos, D., Smirnov, O., Zielinski, M., & van Kasteren, T. L. M. (2017) · Workshop on Data Management for End-to-End Machine Learning (DEEM) · doi:10.1145/3076246.3076248 · Cited by 28 · cited 2×
“Typically, the environment and its conditions are dynamic and, as the systems evolve, the MLOps systems should perform a set of operations to keep the ML models up-to-date [42].”
Gudovskiy, D., Rigazio, L., Ishizaka, S., Kozuka, K., & Tsukizawa, S. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01633 · Cited by 27
Young, Z., & Steele, R. (2022) · International Journal of Information Management Data Insights · doi:10.1016/j.jjimei.2022.100070 · Cited by 26
SensiX++: Bringing MLOps and Multi-tenant Model Serving to Sensory Edge Devices
Min, C., Mathur, A., Acer, U. G., Montanari, A., & Kawsar, F. (2023) · ACM Transactions on Embedded Computing Systems · doi:10.1145/3617507 · Cited by 23 · cited 1×
“Velox automatically monitors the quality of the model evaluated by selected performance metrics.”
Manapragada, C. (2021) · Figshare · doi:10.26180/14585457 · Cited by 22
Manapragada, C. (2021) · Figshare · doi:10.26180/14585457.v1 · Cited by 21
BOAT: A Bayesian Optimization AutoML Time-series Framework for Industrial Applications
Kurian, J. J., Dix, M., Amihai, I., Ceusters, G., & Prabhune, A. (2021) · IEEE International Conference on Big Data Computing Service and Applications (BigDataService) · doi:10.1109/bigdataservice52369.2021.00008 · Cited by 21 · cited 1×
“As discussed in Section 3.2, DataOps involves performing several operations, including data cleaning, ensuring the robustness of the entire ML system to data corruptions such as distributional”
InfluentialMasood, A., & Hashmi, A. (2019) · Apress eBooks · doi:10.1007/978-1-4842-4106-6 · Cited by 20
The Risks of Machine Learning SystemsPreprint
Tan, S., Taeihagh, A., & Baxter, K. K. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2204.09852 · Cited by 20
Did the Model Change? Efficiently Assessing Machine Learning API ShiftsPreprint
Chen, L., Cai, T., Zaharia, M., & Zou, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.14203 · Cited by 19 · cited 1×
“The article discussed existing strategies to standardize trustworthy AI systems.”
Synthesizing labeled data to enhance soft sensor performance in data-scarce regions
Lyu, Y., Chen, J., & Song, Z. (2021) · Control Engineering Practice · doi:10.1016/j.conengprac.2021.104903 · Cited by 18 · cited 1×
“Therefore, incorporating a centralized version control mechanism, a principle derived from continuous integration, is the first step after finalizing a releasable solution [162].”
ModelCI-e: Enabling Continual Learning in Deep Learning Serving SystemsPreprint
Huang, Y., Zhang, H., Wen, Y., Sun, P., & Nguyen Binh Duong, T. A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.03122 · Cited by 16 · cited 2×
“However, finding the best combination of hyperparameter values is not straightforward since it involves pre-defining several attributes, such as hyperparameter search methods and value ranges [62].”
Driftage: a multi-agent system framework for concept drift detection
Vieira, D. M., Fernandes, C. O., de Lucena, C. J. P., & Lifschitz, S. (2021) · GigaScience · doi:10.1093/gigascience/giab030 · Cited by 15
Building Maintainable Software Using Abstraction Layering
Spray, J., Sinha, R., Sen, A., & Cheng, X. (2021) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2021.3119012 · Cited by 13
CMS: A Continuous Machine-Learning and Serving Platform for Industrial Big Data
Li, K., & Gui, N. (2020) · Future Internet · doi:10.3390/fi12060102 · Cited by 12 · cited 2×
“Another considerable challenge for every lifelong learning system, and MLOps in particular, is the change in the probability distribution that represents the relationship between input data and target variables P ( y | X ) , known as concept drift [149].”
Meister, M., Sheikholeslami, S., Payberah, A. H., Vlassov, V., & Dowling, J. (2020) · doi:10.1145/3426745.3431338 · Cited by 12 · cited 1×
“As discussed in Section 3.2, DataOps involves performing several operations, including data cleaning, ensuring the robustness of the entire ML system to data corruptions such as distributional”
InfluentialGreco, S., & Cerquitelli, T. (2021) · IEEE ... International Conference on Data Mining workshops · doi:10.1109/icdmw53433.2021.00049 · Cited by 12 · cited 1×
“In another article [30], Chatila et al. also discussed the requirements to design trustworthy AI systems.”
Witten, I. H., Frank, E., Hall, M. A., & Pal, C. (2016) · Elsevier eBooks · doi:10.1016/b978-0-12-804291-5.00009-x · Cited by 9 · cited 1×
“However, there are other popular performance metrics, such as the area under the receiver operating characteristic curve (AUC-ROC) , the confusion matrix, and the coefficient of determination R 2 [51].”
Testing of machine learning models with limited samples: an industrial vacuum pumping application
Chatterjee, A., Ahmed, B. S., Hallin, E., & Engman, A. (2022) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3540250.3558943 · Cited by 9 · cited 2×
“A recent article published by Wing [175] supports the trustworthiness of AI systems by benefiting from other communities.”
Self-citationDiakonikolas, I., Kamath, G., Kane, D. M., Li, J. Z., Moitra, A., & Stewart, A. (2021) · Communications of the ACM · doi:10.1145/3453935 · Cited by 8 · cited 1×
“Likewise, the robustness property must be considered at the algorithmic level, especially in high dimensions, where computations are relatively expensive [40].”
DataOps-4G: On Supporting Generalists in Data Quality Discovery
Yu, S., Chen, T., Han, L., Demartini, G., & Sadiq, S. W. (2022) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2022.3151605 · Cited by 8 · cited 2×
“The authors dissected some principles and suggestions for building trustworthy AI systems.”
Continual-Learning-as-a-Service (CLaaS): On-Demand Efficient Adaptation of Predictive ModelsPreprint
Semola, R., Lomonaco, V., & Bacciu, D. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2206.06957 · Cited by 8 · cited 2×
“The retrieved approaches are organized according to the robustness aspect addressed in Table 1.”
Shahoud, S., Gunnarsdottir, S., Khalloof, H., Düpmeier, C., & Hagenmeyer, V. (2019) · International Conference on Management of Emergent Digital EcoSystems (MEDES) · doi:10.1145/3297662.3365807 · Cited by 7 · cited 2×
“In another approach, Shahoud et al. proposed a framework for managing ML tasks in big data environments [143].”
The Connection between Out-of-Distribution Generalization and Privacy of ML ModelsPreprint
Mahajan, D., Tople, S., & Sharma, A. K. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.03369 · Cited by 7 · cited 1×
“Another toolkit was built to handle data quality issues, namely DQLearn [144].”
Atwal, H. (2019) · Apress eBooks · doi:10.1007/978-1-4842-5104-1_9 · Cited by 7 · cited 1×
“Since automation involves several steps, ranging from system build-up to system monitoring, the robustness aspect should be addressed at every step.”
Data Science Solutions on Azure
Soh, J., & Singh, P. (2020) · Apress eBooks · doi:10.1007/978-1-4842-6405-8 · Cited by 7
K2E: Building MLOps Environments for Governing Data and Models Catalogues while Tracking Versions
Zárate, G., Miñón, R., Díaz-de-Arcaya, J., & Torre-Bastida, A. I. (2022) · IEEE International Conference on Software Architecture Companion (ICSA-C) · doi:10.1109/icsa-c54293.2022.00047 · Cited by 7 · cited 2×
“Typically, the environment and its conditions are dynamic and, as the systems evolve, the MLOps systems should perform a set of operations to keep the ML models up-to-date [42].”
Bayesian Adaptation for Covariate Shift
Zhou, A., & Levine, S. (2021) · neural information processing systems · Cited by 6
Proceedings of the 1st Workshop on Distributed Machine Learning
(2020) · doi:10.1145/3426745 · Cited by 6
Ease.ml/snoopy: Towards Automatic Feasibility Study for Machine Learning ApplicationsPreprint
Renggli, C., Rimanić, L., Kolar, L., Hollenstein, N., Wu, W., & Zhang, C. (2020) · arXiv (Cornell University) · Cited by 1
Automatic Feasibility Study via Data Quality Analysis for ML: A Case-Study on Label NoisePreprint
Renggli, C., Rimanić, L., Kolar, L., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.08410
Research works printed but not linked: 39
- Hazrat Ali, Christer Grönlund, and Zubair Shah. 2023. Leveraging GANs for data scarcity of COVID-19: Beyond the hype. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 659–667.
- Davide Anguita, Luca Ghelardoni, Alessandro Ghio, Luca Oneto, and Sandro Ridella. 2012. The ‘K’ in K-fold cross validation. In 20th European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning (ESANN). i6doc. com publ, 441–446.
- Firas Bayram, Bestoun S. Ahmed, Erik Hallin, and Anton Engman. 2022. A drift handling approach for self-adaptive ML software in scalable industrial processes. In 2022 37th IEEE/ACM International Conference on Automated Software Engineering (ASE). IEEE.
- Eric Breck, Neoklis Polyzotis, Sudip Roy, Steven Whang, and Martin Zinkevich. 2019. Data validation for machine learning. In MLSys.
- Mandepudi Nobel Chowdary, Bussa Sankeerth, Chennupati Kumar Chowdary, and Manu Gupta. 2022. Accelerating the machine learning model deployment using MLOps. In Journal of Physics: Conference Series, Vol. 2327. IOP Publishing, 012027.
- Abdur Chowdhury and Joshua Alspector. 2003. Data duplication: An imbalance problem?. In ICML’2003 Workshop on Learning from Imbalanced Data Sets (II), Washington, DC. Citeseer.
- European Commission, Content Directorate-General for Communications Networks, and Technology. 2019. Ethics Guidelines for Trustworthy AI. Publications Office.
- Lisa Ehrlinger, Thomas Grubinger, Bence Varga, Mario Pichler, Thomas Natschläger, and Jürgen Zeindl. 2018. Treating missing data in industrial data analytics. In 2018 Thirteenth International Conference on Digital Information Management (ICDIM’18). IEEE, 148–155.
- Xinjian Guo, Yilong Yin, Cailing Dong, Gongping Yang, and Guangtong Zhou. 2008. On the class imbalance problem. In 2008 Fourth International Conference on Natural Computation, Vol. 4. IEEE, 192–201.
- Fabian Hinder, André Artelt, and Barbara Hammer. 2020. Towards non-parametric drift detection via dynamic adapting window independence drift detection (DAWIDD). In International Conference on Machine Learning. PMLR, 4249–4259.
- Hossein Hosseini, Baicen Xiao, and Radha Poovendran. 2017. Google’s Cloud Vision API is not robust to noise. In 2017 16th IEEE International Conference on Machine Learning and Applications (ICMLA’17). IEEE, 101–105.
- Jez Humble and David Farley. 2010. Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation. Pearson Education.
- Davinder Kaur, Suleyman Uslu, and Arjan Durresi. 2020. Requirements for trustworthy artificial intelligence–A review. In International Conference on Network-Based Information Systems. Springer, 105–115.
- Abhishek Kumar, Tristan Braud, Sasu Tarkoma, and Pan Hui. 2020. Trustworthy AI in the age of pervasive computing and big data. In 2020 IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops’20). IEEE, 1–6.
- Robyn Larracy, Angkoon Phinyomark, and Erik Scheme. 2021. Machine learning model validation for early stage studies with small sample sizes. In 2021 43rd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC’21). IEEE, 2314–2319.
- 10.1109/WAIN52551.2021.00024 · DOI · Paper page
- Ankur Mallick, Kevin Hsieh, Behnaz Arzani, and Gauri Joshi. 2022. Matchmaker: Data drift mitigation in machine learning for large-scale systems. Proceedings of Machine Learning and Systems 4 (2022), 77–94.
- Jose Mathew, Meghana Negi, Rutvik Vijjali, and Jairaj Sathyanarayana. 2021. DeFraudNet: An end-to-end weak supervision framework to detect fraud in online food delivery. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases. Springer, 85–99.
- Beatriz M. A. Matsui and Denise H. Goya. 2022. MLOps: A guide to its adoption in the context of responsible AI. In 2022 IEEE/ACM 1st International Workshop on Software Engineering for Responsible Artificial Intelligence (SE4RAI’22). IEEE, 45–49.
- Richard Hugh Moulton, Herna L. Viktor, Nathalie Japkowicz, and João Gama. 2018. Clustering in the presence of concept drift. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases. Springer, 339–355.
- Xiaolei Qian and Gio Wiederhold. 1986. Knowledge-based integrity constraint validation. In VLDB, Vol. 86. Citeseer, 25–28.
- 10.5555/1462129 · doi:10.5555/1462129
- Emmanuel Raj. 2021. Engineering MLOps: Rapidly Build, Test, and Manage Production-ready Machine Learning Life Cycles at Scale. Packt Publishing Ltd.
- Mengye Ren, Wenyuan Zeng, Bin Yang, and Raquel Urtasun. 2018. Learning to reweight examples for robust deep learning. In International Conference on Machine Learning. PMLR, 4334–4343.
- Andras Rozsa, Manuel Günther, and Terrance E. Boult. 2016. Are accuracy and robustness correlated. In 2016 15th IEEE International Conference on Machine Learning and Applications (ICMLA’16). IEEE, 227–232.
- Sebastian Schelter Felix Biessmann Tim Januschowski David Salinas Stephan Seufert and Gyuri Szarvas. 2018. On challenges in machine learning model management. IEEE Data Eng. Bull. 41 (2018) 5–15.
- Shrey Shrivastava, Dhaval Patel, Nianjun Zhou, Arun Iyengar, and Anuradha Bhamidipaty. 2020. DQLearn: A toolkit for structured data quality learning. In 2020 IEEE International Conference on Big Data (Big Data’20). IEEE, 1644–1653.
- Lucas Cardoso Silva, Fernando Rezende Zagatti, Bruno Silva Sette, Lucas Nildaimon dos Santos Silva, Daniel Lucrédio, Diego Furtado Silva, and Helena de Medeiros Caseli. 2020. Benchmarking machine learning solutions in production. In 2020 19th IEEE International Conference on Machine Learning and Applications (ICMLA’20). IEEE, 626–633.
- Masahiro Sugimoto, Masahiro Takada, and Masakazu Toi. 2013. Comparison of robustness against missing values of alternative decision tree and multiple logistic regression for predicting clinical data in primary breast cancer. In 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC’13). IEEE, 3054–3057.
- Georgios Symeonidis, Evangelos Nerantzis, Apostolos Kazakis, and George A. Papakostas. 2022. MLOps-definitions, tools and challenges. In 2022 IEEE 12th Annual Computing and Communication Workshop and Conference (CCWC’22). IEEE, 0453–0460.
- Sunil Thulasidasan, Sushil Thapa, Sayera Dhaubhadel, Gopinath Chennupati, Tanmoy Bhattacharya, and Jeff Bilmes. 2021. An effective baseline for robustness to distributional shift. In 2021 20th IEEE International Conference on Machine Learning and Applications (ICMLA’21). IEEE, 278–285.
- Mark Treveil, Nicolas Omont, Clément Stenac, Kenji Lefevre, Du Phan, Joachim Zentici, Adrien Lavoillotte, Makoto Miyazaki, and Lynn Heidmann. 2020. Introducing MLOps. O’Reilly Media.
- Manasi Vartak, Pablo Ortiz, Kathryn Siegel, Harihar Subramanyam, Samuel Madden, and Matei Zaharia. 2015. Supporting fast iteration in model building. In NIPS Workshop LearningSys. 1–6.
- Robert Woitsch, Wilfrid Utz, Anna Sumereder, Bernhard Dieber, Benjamin Breiling, Laura Crompton, Michael Funk, Karin Bruckmüller, and Stefan Schumann. 2021. Collaborative model-based process assessment for trustworthy AI in robotic platforms. In International Conference on Society 5.0. Springer, 163–174.
- Yinjun Wu, Edgar Dobriban, and Susan Davidson. 2020. DeltaGrad: Rapid retraining of machine learning models. In International Conference on Machine Learning. PMLR, 10355–10366.
- Limin Yang, Wenbo Guo, Qingying Hao, Arridhana Ciptadi, Ali Ahmadzadeh, Xinyu Xing, and Gang Wang. 2021. CADE: Detecting and explaining concept drift samples for security applications. In 30th USENIX Security Symposium (USENIX Security’21). 2327–2344.
- Matei Zaharia, Andrew Chen, Aaron Davidson, Ali Ghodsi, Sue Ann Hong, Andy Konwinski, Siddharth Murching, Tomas Nykodym, Paul Ogilvie, Mani Parkhe, et al. 2018. Accelerating the machine learning lifecycle with MLflow. IEEE Data Eng. Bull. 41, 4 (2018), 39–45.
- Friedemann Zenke, Ben Poole, and Surya Ganguli. 2017. Continual learning through synaptic intelligence. In International Conference on Machine Learning. PMLR, 3987–3995.
- Yizhen Zhao. 2021. Machine Learning in Production: A Literature Review. Universiteit van Amsterdam Technical Report.
Cited by
Data Quality in the Age of AI: A Review of Governance, Ethics, and the FAIR Principles
Guillen-Aguinaga, M., Aguinaga-Ontoso, E., Guillen-Aguinaga, L., Guillen-Grima, F., & Aguinaga-Ontoso, I. (2025) · Data · doi:10.3390/data10120201 · Cited by 37
“…ongoing validation to ensure that training data remain representative of current conditions [18,63,69,181–183, 205 Recent adaptations of the FAIR framework emphasize AI readiness and the need for rigorous attention to provenance, bias detection, and ethical sourcing [25,26,208,236–240].”
Boltaboyeva, A., Baigarayeva, Z., Иманбек, Б., Ozhikenov, К., Getahun, A. J., Aidarova, T., & Karymsakova, N. (2025) · Applied Sciences · doi:10.3390/app15126840 · Cited by 23
Large Language Models for Constructing and Optimizing Machine Learning Workflows: A Survey
Gu, Y., You, H., Cao, J., Yu, M., Fan, H., & Qian, S. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3773084 · Cited by 63
Deep learning for sustainable development across climate, energy, agriculture and urban systems
Sharma, H., & Kaur, S. (2025) · Discover Sustainability · doi:10.1007/s43621-025-02186-6 · Cited by 14
“For example, combining aerial drone footage with soil sensor data enhances agricultural monitoring, while fusing satellite data with policy documents or disaster reports supports contextualized emergency response [10, 18].”
InfluentialSu, Z., Zhang, L., Chen, K., Zhi, L., Huan, X., Chen, F., Wang, X., & Wang, J. (2025) · Gondwana Research · doi:10.1016/j.gr.2025.11.021 · Cited by 5
Metadata
- Authors
- Bayram, F., & Ahmed, B. S.
- Affiliations
- Karlstad University (SE)
- Venue
- ACM Computing Surveys
- Published
- 2025-01-22
- Volume
- 57
- Issue
- 5
- Pages
- 1-35
- ISSN
- 0360-0300 (ISSN-L)
- 1557-7341
- OpenAlex
- W4405547182
- Cited by
- 48
- References
- 174
- Citations by year
- 2026: 31 · 2025: 16
- FWCI
- 7.05
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- ACM Computing Surveys (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion, cc-by)
- Digitala vetenskapliga arkivet (Diva) (Karlstad University) (submittedVersion, other-oa)
- arXiv (Cornell University) (cc-by)
- Funders
- Knowledge Foundation of Sweden (KKS) through the Synergy Project AIDA - A Holistic AI-driven Networking and Processing Framework for Industrial IoT (Rek:20200067)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Computer Networks and Communications › Software System Performance and Reliability
- Keywords
- MLOps, machine learning robustness, trustworthy AI, machine learning in production
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.