AI system engineering key challenges and lessons learned
Open accessAbstract
The main challenges are discussed together with the lessons learned from past and ongoing research along the development cycle of machine learning systems. This will be done by taking into account intrinsic conditions of nowadays deep learning models, data and software quality issues and human-centered artificial intelligence (AI) postulates, including confidentiality and ethical aspects. The analysis outlines a fundamental theory-practice gap which superimposes the challenges of AI system engineering at the level of data quality assurance, model building, software engineering and deployment. The aim of this paper is to pinpoint research topics to explore approaches to address these challenges.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
These pipelines need to handle code, data, the model itself, and large system-level complexity.
“These pipelines are well established in traditional software development however need more research when adapting them to AI models because these pipelines not only need to handle code but also data and the model itself in addition to a large system-level complexity [7, I18].”
From The pipeline for the continuous development of artificial intelligence models—Current state of research and practice (2023) · cites this paper as [7]
Automated end-to-end CI/CD lifecycle pipelines are well established in traditional software development.
“These pipelines are well established in traditional software development however need more research when adapting them to AI models because these pipelines not only need to handle code but also data and the model itself in addition to a large system-level complexity [7, I18].”
From The pipeline for the continuous development of artificial intelligence models—Current state of research and practice (2023) · cites this paper as [7]
These pipelines need more research when adapting them to AI models.
“These pipelines are well established in traditional software development however need more research when adapting them to AI models because these pipelines not only need to handle code but also data and the model itself in addition to a large system-level complexity [7, I18].”
From The pipeline for the continuous development of artificial intelligence models—Current state of research and practice (2023) · cites this paper as [7]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404 · see citing works
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126 · see citing works
He, K., Gkioxari, G., Dollar, P., & Girshick, R. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.322 · Cited by 29,137 · see citing works
The Elements of Statistical Learning
Hastie, T., Tibshirani, R., & Friedman, J. (2009) · Springer Series in Statistics · doi:10.1007/978-0-387-84858-7 · Cited by 25,970 · see citing works
Pan, S. J., & Yang, Q. (2010) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2009.191 · Cited by 23,387 · see citing works
Explaining and Harnessing Adversarial Examples
Goodfellow, I. J., Shlens, J., & Szegedy, C. (2014) · arXiv · doi:10.48550/arxiv.1412.6572 · Cited by 10,381 · see citing works
Statistical Analysis with Missing Data
Little, R. J. A., & Rubin, D. B. (2002) · Wiley Series in Probability and Statistics · doi:10.1002/9781119013563 · Cited by 8,473 · see citing works
Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
Xu, K., Ba, J., Kiros, R., Cho, K., Courville, A., Salakhutdinov, R., Zemel, R., & Bengio, Y. (2015) · arXiv · doi:10.48550/arxiv.1502.03044 · Cited by 8,248 · see citing works
Intriguing properties of neural networks
Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., & Fergus, R. (2013) · arXiv · doi:10.48550/arxiv.1312.6199 · Cited by 8,143 · see citing works
Dissecting racial bias in an algorithm used to manage the health of populations
Obermeyer, Z., Powers, B., Vogeli, C., & Mullainathan, S. (2019) · Science · doi:10.1126/science.aax2342 · Cited by 7,137 · cited 1× · see citing works
“The necessity to obtain not only increasing prediction accuracy but also the interpretation of the solutions determined by ML or Deep Learning arises at the latest with the ethical [65,66], legal [67], psychological [68], medical [69,70], and sociological [71] questions tied to their application.”
BackgroundDropout as a bayesian approximation: Representing model uncertainty in deep learning
Gal, Y., & Ghahramani, Z. (2015) · arXiv · doi:10.48550/arxiv.1506.02142 · Cited by 6,471 · see citing works
mixup: Beyond Empirical Risk Minimization
Zhang, H., Cisse, M., Dauphin, Y. N., & Lopez-Paz, D. (2017) · arXiv · doi:10.48550/arxiv.1710.09412 · Cited by 5,831 · see citing works
The Algorithmic Foundations of Differential Privacy
Dwork, C., & Roth, A. (2014) · Foundations and Trends® in Theoretical Computer Science · doi:10.1561/0400000042 · Cited by 5,557 · see citing works
CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features
Yun, S., Han, D., Oh, S. J., Chun, S., Choe, J., & Yoo, Y. (2019) · arXiv · doi:10.48550/arxiv.1905.04899 · Cited by 5,439 · see citing works
The mythos of model interpretability
Lipton, Z. C. (2018) · Communications of the ACM · doi:10.1145/3233231 · Cited by 5,427 · see citing works
CutMix: Regularization Strategy to Train Strong Classifiers With Localizable Features
Yun, S., Han, D., Chun, S., Oh, S. J., Yoo, Y., & Choe, J. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00612 · Cited by 5,400 · see citing works
Adversarial examples in the physical world
Kurakin, A., Goodfellow, I. J., & Bengio, S. (2018) · Artificial Intelligence Safety and Security · doi:10.1201/9781351251389-8 · Cited by 5,375 · see citing works
Membership Inference Attacks Against Machine Learning Models
Shokri, R., Stronati, M., Song, C., & Shmatikov, V. (2017) · 2017 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp.2017.41 · Cited by 4,754 · see citing works
Beyond Accuracy: What Data Quality Means to Data Consumers
Wang, R. Y., & Strong, D. M. (1996) · Journal of Management Information Systems · doi:10.1080/07421222.1996.11518099 · Cited by 4,672 · see citing works
Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference
Jacob, B., Kligys, S., Chen, B., Zhu, M., Tang, M., Howard, A., Adam, H., & Kalenichenko, D. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00286 · Cited by 4,114 · see citing works
The Algorithmic Foundations of Differential Privacy
Dwork, C., & Roth, A. (2013) · doi:10.1561/9781601988195 · Cited by 3,856 · see citing works
Deep CORAL: Correlation Alignment for Deep Domain Adaptation
Sun, B., & Saenko, K. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-49409-8_35 · Cited by 3,757 · see citing works
MixMatch: A Holistic Approach to Semi-Supervised LearningPreprint
Berthelot, D., Carlini, N., Goodfellow, I., Papernot, N., Oliver, A., & Raffel, C. A. (2019) · Neural Information Processing Systems · doi:10.48550/arxiv.1905.02249 · Cited by 3,744 · see citing works
On calibration of modern neural networks
Guo, C., Pleiss, G., Sun, Y., & Weinberger, K. Q. (2017) · arXiv · doi:10.48550/arxiv.1706.04599 · Cited by 3,435 · see citing works
Towards A Rigorous Science of Interpretable Machine Learning
Doshi-Velez, F., & Kim, B. (2017) · arXiv · doi:10.48550/arxiv.1702.08608 · Cited by 3,121 · see citing works
Model Inversion Attacks that Exploit Confidence Information and Basic Countermeasures
Fredrikson, M., Jha, S., & Ristenpart, T. (2015) · Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/2810103.2813677 · Cited by 2,961 · see citing works
Understanding deep learning requires rethinking generalization
Zhang, C., Bengio, S., Hardt, M., Recht, B., & Vinyals, O. (2016) · arXiv · doi:10.48550/arxiv.1611.03530 · Cited by 2,838 · see citing works
Knowledge Graph Embedding: A Survey of Approaches and Applications
Wang, Q., Mao, Z., Wang, B., & Guo, L. (2017) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2017.2754499 · Cited by 2,654 · see citing works
Explaining Explanations: An Overview of Interpretability of Machine Learning
Gilpin, L. H., Bau, D., Yuan, B. Z., Bajwa, A., Specter, M., & Kagal, L. (2018) · 2018 IEEE 5th International Conference on Data Science and Advanced Analytics (DSAA) · doi:10.1109/dsaa.2018.00018 · Cited by 2,507 · see citing works
The Effectiveness of Data Augmentation in Image Classification using Deep Learning
Perez, L., & Wang, J. (2017) · arXiv · doi:10.48550/arxiv.1712.04621 · Cited by 2,499 · see citing works
Guidelines for Human-AI Interaction
Amershi, S., Weld, D., Vorvoreanu, M., Fourney, A., Nushi, B., Collisson, P., Suh, J., Iqbal, S., Bennett, P. N., Inkpen, K., et al. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300233 · Cited by 2,439 · see citing works
A Comprehensive Survey of Graph Embedding: Problems, Techniques, and Applications
Cai, H., Zheng, V. W., & Chang, K. C.-C. (2018) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2018.2807452 · Cited by 2,034 · see citing works
Causability and explainability of artificial intelligence in medicine
Holzinger, A., Langs, G., Denk, H., Zatloukal, K., & Müller, H. (2019) · WIREs Data Mining and Knowledge Discovery · doi:10.1002/widm.1312 · Cited by 1,885 · see citing works
Implementing Machine Learning in Health Care — Addressing Ethical Challenges
Char, D. S., Shah, N. H., & Magnus, D. (2018) · New England Journal of Medicine · doi:10.1056/nejmp1714229 · Cited by 1,857 · see citing works
Machine Learning Interpretability: A Survey on Methods and Metrics
Carvalho, D. V., Pereira, E. M., & Cardoso, J. S. (2019) · Electronics · doi:10.3390/electronics8080832 · Cited by 1,842 · see citing works
Nasr, M., Shokri, R., & Houmansadr, A. (2019) · 2019 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp.2019.00065 · Cited by 1,760 · see citing works
A Review of Relational Machine Learning for Knowledge Graphs
Nickel, M., Murphy, K. P., Tresp, V., & Gabrilovich, E. (2015) · Proceedings of the IEEE · doi:10.1109/jproc.2015.2483592 · Cited by 1,684 · see citing works
Wild patterns: Ten years after the rise of adversarial machine learning
Biggio, B., & Roli, F. (2018) · Pattern Recognition · doi:10.1016/j.patcog.2018.07.023 · Cited by 1,563 · see citing works
Dataset Shift in Machine Learning
(2008) · doi:10.7551/mitpress/9780262170055.001.0001 · Cited by 1,553 · see citing works
Hyperband: A Novel Bandit-Based Approach to Hyperparameter OptimizationPreprint
Li, L., Jamieson, K., DeSalvo, G., Rostamizadeh, A., & Talwalkar, A. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1603.06560 · Cited by 1,368 · see citing works
Multimodal Data Fusion: An Overview of Methods, Challenges, and Prospects
Lahat, D., Adali, T., & Jutten, C. (2015) · Proceedings of the IEEE · doi:10.1109/jproc.2015.2460697 · Cited by 1,296 · see citing works
Big data deep learning: Challenges and perspectives
Chen, X.-W., & Lin, X. (2014) · IEEE Access · doi:10.1109/access.2014.2325029 · Cited by 1,255 · see citing works
Knowledge graph refinement: A survey of approaches and evaluation methods
Paulheim, H. (2016) · Semantic Web · doi:10.3233/sw-160218 · Cited by 1,253 · see citing works
Simultaneous Deep Transfer Across Domains and Tasks
Tzeng, E., Hoffman, J., Darrell, T., & Saenko, K. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.463 · Cited by 1,203 · see citing works
Explainable artificial intelligence: A survey
Dosilovic, F. K., Brcic, M., & Hlupic, N. (2018) · 2018 41st International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO) · doi:10.23919/mipro.2018.8400040 · Cited by 1,147 · see citing works
Samek, W., Wiegand, T., & Müller, K.-R. (2017) · arXiv · doi:10.48550/arxiv.1708.08296 · Cited by 1,080 · see citing works
Generating Adversarial Examples with Adversarial Networks
Xiao, C., Li, B., Zhu, J.-Y., He, W., Liu, M., & Song, D. X. (2018) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2018/543 · Cited by 988 · see citing works
Synthesizing Robust Adversarial ExamplesPreprint
Athalye, A., Engstrom, L., Ilyas, A., & Kwok, K. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.07397 · Cited by 973 · see citing works
Hyperband: a novel bandit-based approach to hyperparameter optimization
Li, L., Jamieson, K., DeSalvo, G., Rostamizadeh, A., & Talwalkar, A. (2017) · Journal of Machine Learning Research · Cited by 930 · see citing works
Manifold Mixup: Better Representations by Interpolating Hidden StatesPreprint
Verma, V., Lamb, A., Beckham, C., Najafi, A., Mitliagkas, I., Courville, A. C., López-Paz, D., & Bengio, Y. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.05236 · Cited by 919 · see citing works
(2015) · neural information processing systems · Cited by 912 · see citing works
Interactive machine learning for health informatics: when do we need the human-in-the-loop?
Holzinger, A. (2016) · Brain Informatics · doi:10.1007/s40708-016-0042-6 · Cited by 906 · see citing works
Artificial Intelligence and Black‐Box Medical Decisions: <i>Accuracy versus Explainability</i>
London, A. J. (2019) · Hastings Center Report · doi:10.1002/hast.973 · Cited by 801 · see citing works
Data Augmentation Generative Adversarial NetworksPreprint
Antoniou, A., Storkey, A., & Edwards, H. S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.04340 · Cited by 797 · see citing works
Instance Weighting for Domain Adaptation in NLP
Jiang, J., & Zhai, C. (2007) · Cited by 780 · see citing works
AI can be sexist and racist — it’s time to make it fair
Zou, J., & Schiebinger, L. (2018) · Nature · doi:10.1038/d41586-018-05707-8 · Cited by 765 · see citing works
Hyper-Parameter Optimization: A Review of Algorithms and ApplicationsPreprint
Yu, T., & Zhu, H. C. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.05689 · Cited by 729 · see citing works
A survey of symbolic execution techniques
Baldoni, R., Coppa, E., D’elia, D. C., Demetrescu, C., & Finocchi, I. (2019) · ACM Computing Surveys · doi:10.1145/3182657 · Cited by 708 · see citing works
An orchestrated survey of methodologies for automated software test case generation
Anand, S., Burke, E. K., Chen, T. Y., Clark, J., Cohen, M. B., Grieskamp, W., Harman, M., Harrold, M. J., McMinn, P., Bertolino, A., et al. (2013) · Journal of Systems and Software · doi:10.1016/j.jss.2013.02.061 · Cited by 669 · see citing works
Challenges in deploying machine learning: A survey of case studies
Paleyes, A., Urma, R.-G., & Lawrence, N. D. (2023) · ACM Computing Surveys · doi:10.1145/3533378 · Cited by 638 · see citing works
The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks
Carlini, N., Liu, C., Erlingsson, Ú., Kos, J., & Song, D. (2018) · arXiv · doi:10.48550/arxiv.1802.08232 · Cited by 629 · see citing works
A Survey of Methods for Explaining Black Box Models
Guidotti, R., Monreale, A., Ruggieri, S., Turini, F., Giannotti, F., & Pedreschi, D. (2019) · ACM Computing Surveys · doi:10.1145/3236009 · Cited by 629 · see citing works
Aggarwal, C. C. (2016) · Springer eBooks · doi:10.1007/978-3-319-47578-3 · Cited by 616 · see citing works
Central Moment Discrepancy (CMD) for Domain-Invariant Representation LearningPreprint
Zellinger, W., Grubinger, T., Lughofer, E. D., Natschläger, T., & Saminger‐Platz, S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1702.08811 · Cited by 563 · see citing works
Federated Optimization:Distributed Optimization Beyond the DatacenterPreprint
Konečný, J., McMahan, B., & Ramage, D. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.03575 · Cited by 547 · see citing works
DARPA's explainable artificial intelligence (XAI) program
Gunning, D. (2019) · Proceedings of the 24th International Conference on Intelligent User Interfaces · doi:10.1145/3301275.3308446 · Cited by 546 · see citing works
Deep Multimodal Representation Learning: A Survey
Guo, W., Wang, J., & Wang, S. (2019) · IEEE Access · doi:10.1109/access.2019.2916887 · Cited by 535 · see citing works
The limits and potentials of deep learning for robotics
Sünderhauf, N., Brock, O., Scheirer, W. J., Hadsell, R. T., Fox, D., Leitner, J., Upcroft, B., Abbeel, P., Burgard, W., Milford, M. J., et al. (2018) · QUT ePrints (Queensland University of Technology) · Cited by 519 · see citing works
Metrics for explainable AI: challenges and prospects
Hoffman, R. R., Mueller, S. T., Klein, G., & Litman, J. (2018) · arXiv · doi:10.48550/arxiv.1812.04608 · Cited by 518 · see citing works
Ganju, K., Wang, Q., Yang, W., Gunter, C. A., & Borisov, N. (2018) · Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3243734.3243834 · Cited by 475 · cited 1× · see citing works
“In contrast, the goal of property inference attacks is not at the individual level (membership of a certain class), but rather at the level of aggregated properties of the training data such as amount of data [51].”
BackgroundDefect prediction from static code features: current results, limitations, new approaches
Menzies, T., Milton, Z., Turhan, B., Čukić, B., Jiang, Y., & Bener, A. (2010) · Automated Software Engineering · doi:10.1007/s10515-010-0069-5 · Cited by 462 · see citing works
Hein, M., Andriushchenko, M., & Bitterwolf, J. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00013 · Cited by 444 · see citing works
Medical Image Computing and Computer-Assisted Intervention -- MICCAI 2015
Navab, N., Joachim Hornegger, A., Wells, W. M., & Frangi, A. F. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24571-3 · Cited by 425 · see citing works
Towards a Definition of Knowledge Graphs.
Ehrlinger, L., & Wöß, W. (2016) · Cited by 410 · see citing works
Holzinger, A., Carrington, A., & Müller, H. (2020) · KI - Künstliche Intelligenz · doi:10.1007/s13218-020-00636-z · Cited by 406 · see citing works
Demystifying Membership Inference Attacks in Machine Learning as a Service
Truex, S., Liu, L., Gürsoy, M. E., Yu, L., & Wei, W. (2019) · IEEE Transactions on Services Computing · doi:10.1109/tsc.2019.2897554 · Cited by 389 · see citing works
A systematic approach to exploring embedded system architectures at multiple abstraction levels
Pimentel, A. D., Erbas, C., & Polstra, S. (2006) · IEEE Transactions on Computers · doi:10.1109/tc.2006.16 · Cited by 380 · see citing works
Next-Generation Big Data Analytics: State of the Art, Challenges, and Future Research Topics
Lv, Z., Song, H. H., Val, P. B., Steed, A. J., & Jo, M. (2017) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2017.2650204 · Cited by 366 · see citing works
Industry-scale knowledge graphs
Noy, N. F., Gao, Y., Jain, A. N., Narayanan, A., Patterson, A., & Taylor, J. (2019) · Communications of the ACM · doi:10.1145/3331166 · Cited by 363 · see citing works
Deep CORAL: Correlation Alignment for Deep Domain AdaptationPreprint
Sun, B., & Saenko, K. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1607.01719 · Cited by 361 · see citing works
A comprehensive study on deep learning bug characteristics
Islam, J., Nguyen, G., Pan, R., & Rajan, H. (2019) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3338906.3338955 · Cited by 315 · see citing works
Transferring GANs: Generating Images from Limited Data
Wang, Y., Wu, C., Herranz, L., van de Weijer, J., González-García, A., & Raducanu, B. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01231-1_14 · Cited by 279 · see citing works
Simultaneous Deep Transfer Across Domains and Tasks
Hoffman, J., Tzeng, E., Darrell, T., & Saenko, K. (2017) · Advances in computer vision and pattern recognition · doi:10.1007/978-3-319-58347-1_9 · Cited by 277 · see citing works
Benchmarking TPU, GPU, and CPU Platforms for Deep LearningPreprint
Wang, Y. E., Wei, G.-Y., & Brooks, D. M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.10701 · Cited by 236 · see citing works
Generating Adversarial Examples with Adversarial NetworksPreprint
Xiao, C., Li, B., Zhu, J.-Y., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1801.02610 · Cited by 230 · see citing works
Explanatory Preferences Shape Learning and Inference
Lombrozo, T. (2016) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2016.08.001 · Cited by 229 · see citing works
Nonlinear Model Predictive Control
Grancharova, A., & Johansen, T. A. (2012) · Lecture notes in control and information sciences · doi:10.1007/978-3-642-28780-0_2 · Cited by 227 · see citing works
Holzinger, A., Kieseberg, P., Weippl, E. R., & Tjoa, A. M. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-99740-7_1 · Cited by 210 · see citing works
Quality analysis of source code comments
Steidl, D., Hummel, B., & Juergens, E. (2013) · International Conference on Program Comprehension · doi:10.1109/icpc.2013.6613836 · Cited by 200 · see citing works
Data Curation at Scale: The Data Tamer System
Stonebraker, M. R., Beskales, G., Pagan, A., Bruckner, D., Cherniack, M., Xu, S., Analytics, V., Ilyas, I. F., & Zdonik, S. (2013) · Cited by 190 · see citing works
Data Validation for Machine Learning
Breck, E. J., Zinkevich, M., Polyzotis, N., Whang, S. E., & Roy, S. (2019) · Cited by 182 · see citing works
Understanding Membership Inferences on Well-Generalized Learning Models
Long, Y., Bindschaedler, V., Wang, L., Bu, D., Wang, X., Tang, H., Gunter, C. A., & Chen, K. (2018) · arXiv · doi:10.48550/arxiv.1802.04889 · Cited by 166 · see citing works
Using and understanding cross-validation strategies. Perspectives on Saeb et al.
Little, M. A., Varoquaux, G. P., Saeb, S., Lonini, L., Jayaraman, A., Mohr, D. C., & Kording, K. P. (2017) · GigaScience · doi:10.1093/gigascience/gix020 · Cited by 159 · see citing works
No AI Is an Island: The Case for Teaming Intelligence
Johnson, M., & Vera, A. H. (2019) · AI Magazine · doi:10.1609/aimag.v40i1.2842 · Cited by 138 · see citing works
Blessing of dimensionality: mathematical foundations of the statistical physics of data
Gorban, A. N., & Tyukin, I. (2018) · Figshare · Cited by 137 · see citing works
Forcier, M. B., Gallois, H., Mullan, S. M., & Joly, Y. (2019) · Journal of Law and the Biosciences · doi:10.1093/jlb/lsz013 · Cited by 135 · see citing works
Blessing of dimensionality: mathematical foundations of the statistical physics of data
Gorban, A. N., & Tyukin, I. Y. (2018) · Philosophical Transactions of the Royal Society A Mathematical Physical and Engineering Sciences · doi:10.1098/rsta.2017.0237 · Cited by 128 · see citing works
Mathematics of Deep LearningPreprint
Vidal, R., Bruna, J., Giryes, R., & Soatto, S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1712.04741 · Cited by 126 · see citing works
Explicit Nonlinear Model Predictive Control
Grancharova, A., & Johansen, T. A. (2012) · Lecture notes in control and information sciences · doi:10.1007/978-3-642-28780-0 · Cited by 115 · see citing works
Domain-Invariant Partial-Least-Squares Regression
Nikzad‐Langerodi, R., Zellinger, W., Lughofer, E. D., & Saminger‐Platz, S. (2018) · Analytical Chemistry · doi:10.1021/acs.analchem.8b00498 · Cited by 110 · see citing works
Requirements for Data Quality Metrics
Heinrich, B., Hristova, D., Klier, M., Schiller, A., & Szubartowicz, M. (2017) · Journal of Data and Information Quality · doi:10.1145/3148238 · Cited by 108 · see citing works
Entity Resolution and Information Quality
(2011) · Elsevier eBooks · doi:10.1016/c2009-0-63396-1 · Cited by 102 · see citing works
Robust unsupervised domain adaptation for neural networks via moment alignment
Zellinger, W., Moser, B., Grubinger, T., Lughofer, E. D., Natschläger, T., & Saminger‐Platz, S. (2019) · Information Sciences · doi:10.1016/j.ins.2019.01.025 · Cited by 98 · see citing works
Classifying Code Comments in Java Open-Source Software Systems
Pascarella, L., & Bacchelli, A. (2017) · IEEE/ACM International Conference on Mining Software Repositories (MSR) · doi:10.1109/msr.2017.63 · Cited by 93 · see citing works
Domain adaptation for regression under Beer–Lambert’s law
Nikzad‐Langerodi, R., Zellinger, W., Saminger‐Platz, S., & Moser, B. (2020) · Knowledge-Based Systems · doi:10.1016/j.knosys.2020.106447 · Cited by 81 · see citing works
An annotated fluorescence image dataset for training nuclear segmentation methods
Kromp, F., Bozsaky, E., Rifatbegovic, F., Fischer, L., Ambros, M., Berneder, M., Weiss, T., Lazic, D., Dörr, W., Hanbury, A., et al. (2020) · Scientific Data · doi:10.1038/s41597-020-00608-w · Cited by 77 · see citing works
The Judicial Demand for Explainable Artificial IntelligencePreprint
Deeks, A. S. (2019) · SSRN Electronic Journal · Cited by 74 · see citing works
Meloni, P., Capotondi, A., Deriu, G., Brian, M., Conti, F., Rossi, D., Raffo, L., & Luca Benini, C. (2018) · ACM Transactions on Reconfigurable Technology and Systems · doi:10.1145/3284357 · Cited by 73 · see citing works
GRT: Program-Analysis-Guided Random Testing (T)
Ma, L., Artho, C. V., Zhang, C., Satō, H., Gmeiner, J., & Ramler, R. (2015) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase.2015.49 · Cited by 72 · see citing works
Introduction to MAchine Learning & Knowledge Extraction (MAKE)
Holzinger, A. (2017) · Machine Learning and Knowledge Extraction · doi:10.3390/make1010001 · Cited by 65 · see citing works
Integrating risk-based testing in industrial test processes
Felderer, M., & Ramler, R. (2013) · Software Quality Journal · doi:10.1007/s11219-013-9226-y · Cited by 60 · see citing works
Introduction: Measuring Data Quality for Ongoing Improvement
Sebastian-Coleman, L. (2013) · Elsevier eBooks · doi:10.1016/b978-0-12-397033-6.00020-1 · Cited by 57 · see citing works
Mixture Density Generative Adversarial Networks
Eghbalzadeh, H., Zellinger, W., & Widmer, G. (2018) · Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2019.00597 · Cited by 45 · see citing works
Detection of Disseminated Tumor Cells in Neuroblastoma
Méhes, G., Luegmayr, A., Kornmüller, R., Ambros, I. M., Ladenstein, R. L., Gadner, H., & Ambros, P. F. (2003) · American Journal Of Pathology · doi:10.1016/s0002-9440(10)63669-3 · Cited by 42 · see citing works
A DaQL to Monitor Data Quality in Machine Learning Applications
Ehrlinger, L., Haunschmid, V., Palazzini, D., & Lettner, C. (2019) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-27615-7_17 · Cited by 40 · see citing works
ASYMPTOTIC OPTIMALITY AND EFFICIENT COMPUTATION OF THE LEAVE-SUBJECT-OUT CROSS-VALIDATIONPreprint
Xu, G., & Huang, J. Z. (2016) · Cited by 39 · see citing works
Asymptotic optimality and efficient computation of the leave-subject-out cross-validation
Xu, G., & Huang, J. Z. (2012) · The Annals of Statistics · doi:10.1214/12-aos1063 · Cited by 38 · see citing works
The limits and potentials of deep learning for robotics
Sünderhauf, N., Brock, O., Scheirer, W., Hadsell, R., Fox, D., Leitner, J., Upcroft, B., Abbeel, P., Burgard, W., Milford, M., et al. (2018) · The International Journal of Robotics Research · doi:10.1177/0278364918770733 · Cited by 38 · see citing works
Fusion of transfer learning features and its application in image classification
Akilan, T., Wu, Q. M. J., Yang, Y., & Safaei, A. (2017) · Conference proceedings - Canadian Conference on Electrical and Computer Engineering · doi:10.1109/ccece.2017.7946733 · Cited by 33 · see citing works
Analyzing Code Comments to Boost Program Comprehension
Shinyama, Y., Arahori, Y., & Gondow, K. (2018) · Proceedings - Asia Pacific Software Engineering Conference/Proceedings · doi:10.1109/apsec.2018.00047 · Cited by 32 · see citing works
Industrial Federated Learning – Requirements and System Design
Hießl, T., Schall, D., Kemnitz, J., & Schulte, S. (2020) · Communications in computer and information science · doi:10.1007/978-3-030-51999-5_4 · Cited by 30 · see citing works
Multi-source transfer learning of time series in cyclical manufacturing
Zellinger, W., Grubinger, T., Zwick, M., Lughofer, E. D., Schöner, H., Natschläger, T., & Saminger‐Platz, S. (2019) · Journal of Intelligent Manufacturing · doi:10.1007/s10845-019-01499-4 · Cited by 29 · see citing works
Treating Missing Data in Industrial Data Analytics
Ehrlinger, L., Grubinger, T., Varga, B., Pichler, M., Natschläger, T., & Zeindl, J. (2018) · International Conference on Digital Information Management · doi:10.1109/icdim.2018.8846984 · Cited by 26 · see citing works
A Survey of Data Quality Measurement and Monitoring Tools
Ehrlinger, L., & Wöß, W. (2022) · Frontiers in Big Data · doi:10.3389/fdata.2022.850611 · Cited by 25 · see citing works
Self-citationDeriving an Optimal Noise Adding Mechanism for Privacy-Preserving Machine Learning
Kumar, M., Rossbory, M., Moser, B., & Freudenthaler, B. (2019) · Communications in computer and information science · doi:10.1007/978-3-030-27684-3_15 · Cited by 24 · see citing works
Adapting automated test generation to GUI testing of industry applications
Ramler, R., Buchgeher, G., & Klammer, C. (2017) · Information and Software Technology · doi:10.1016/j.infsof.2017.07.005 · Cited by 23 · see citing works
Beyond federated learning: On confidentiality-critical machine learning applications in industry
Zellinger, W., Wieser, V., Kumar, M., Brunner, D. J., Shepeleva, N., Gálvez, R., Langer, J., Fischer, L., & Moser, B. (2021) · Procedia Computer Science · doi:10.1016/j.procs.2021.01.296 · Cited by 23 · see citing works
The Tradeoff Between Privacy and Accuracy in Anomaly Detection Using Federated XGBoostPreprint
Yang, M., Song, L., Xu, J., Li, C., & Tan, G. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.07157 · Cited by 22 · see citing works
Transferring GANs: generating images from limited dataPreprint
Wang, Y., Wu, C., Herranz, L., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.01677 · Cited by 21 · see citing works
(2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01270-0 · Cited by 21 · see citing works
RbG: A documentation generator for scientific and engineering software
Moser, M., Pichler, J., Fleck, G., & Witlatschil, M. (2015) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner.2015.7081857 · Cited by 20 · see citing works
Optimal Accuracy-Privacy Trade-Off for Secure Computations
Ah-Fat, P., & Huth, M. (2018) · IEEE Transactions on Information Theory · doi:10.1109/tit.2018.2886458 · Cited by 19 · see citing works
A Practical Guide to Entity Resolution with OYSTER
Talburt, J. R., & Zhou, Y. (2013) · Springer eBooks · doi:10.1007/978-3-642-36257-6_11 · Cited by 17 · see citing works
Trusted AI and the Contribution of Trust Modeling in Multiagent Systems
Cohen, R. A., Schaekermann, M., Liu, S., & Cormier, M. (2019) · Adaptive Agents and Multi-Agents Systems · doi:10.65109/dzex3063 · Cited by 17 · see citing works
Applying AI in Practice: Key Challenges and Lessons Learned
Fischer, L., Ehrlinger, L., Geist, V., Ramler, R., Sobieczky, F., Zellinger, W., & Moser, B. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-57321-8_25 · Cited by 17 · see citing works
Multi-language re-documentation to support a COBOL to Java migration project
Dorninger, B., Moser, M., & Pichler, J. (2017) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner.2017.7884669 · Cited by 16 · see citing works
An optimal (∊,δ) -differentially private learning of distributed deep fuzzy models
Kumar, M., Rossbory, M., Moser, B., & Freudenthaler, B. (2020) · Information Sciences · doi:10.1016/j.ins.2020.07.044 · Cited by 15 · see citing works
Jung, C., & Kim, C. (2014) · Cytometry Part A · doi:10.1002/cyto.a.22467 · Cited by 14 · see citing works
Ramler, R., & Wolfmaier, K. (2008) · Proceedings of the ... ACM-IEEE International Symposium on Empirical Software Engineering and Measurement/International Symposium on Empirical Software Engineering and Measurement · doi:10.1145/1414004.1414071 · Cited by 14 · see citing works
Modeling of Deep Neural Network (DNN) Placement and Inference in Edge Computing
Bensalem, M., Dizdarević, J., & Jukan, A. (2020) · IEEE International Conference on Communications workshops · doi:10.1109/iccworkshops49005.2020.9145449 · Cited by 14 · see citing works
A Process for Risk-Based Test Strategy Development and Its Industrial Evaluation
Ramler, R., & Felderer, M. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-26844-6_26 · Cited by 13 · see citing works
Automated Continuous Data Quality Measurement with QuaIIe
Ehrlinger, L., Werth, B., & Wöß, W. (2018) · International Journal On Advances in Software · Cited by 13 · see citing works
Talburt, J. R. (2010) · Elsevier eBooks · doi:10.1016/b978-0-12-381972-7.00005-1 · Cited by 12 · see citing works
Redocumentation of a legacy banking system
Geet, J. V., Ebraert, P., & Demeyer, S. (2010) · doi:10.1145/1862372.1862382 · Cited by 12 · see citing works
A Novel Data Quality Metric for Minimality
Ehrlinger, L., & Wöß, W. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-19143-6_1 · Cited by 12 · see citing works
Optimization and deployment of CNNs at the edge
Meloni, P., Loi, D., Busia, P., Deriu, G., Pimentel, A. D., Sapra, D., Stefanov, T. P., Minakova, S., Conti, F., Luca Benini, C., et al. (2019) · ACM International Conference on Computing Frontiers (CF) · doi:10.1145/3310273.3323435 · Cited by 12 · see citing works
Automating Test Reuse for Highly Configurable Software
Fischer, S., Ramler, R., Linsbauer, L., & Egyed, A. F. (2019) · International Software Product Line Conference (SPLC) · doi:10.1145/3336294.3336305 · Cited by 12 · see citing works
Leveraging Machine Learning for Software Redocumentation
Geist, V., Moser, M., Pichler, J., Beyer, S., & Pinzger, M. (2020) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner48275.2020.9054838 · Cited by 12 · see citing works
Pluggable Analysis Viewpoints for Design Space Exploration
Masin, M., Limonad, L., Sela, A., Boaz, D., Greenberg, L., Mashkif, N., & Rinat, R. (2013) · Procedia Computer Science · doi:10.1016/j.procs.2013.01.024 · Cited by 11 · see citing works
Improving data quality and closing data gaps with machine learning
Cagala, T. (2017) · IFC Bulletins chapters · Cited by 11 · see citing works
Differentially Private Learning of Distributed Deep Models
Kumar, M., Rossbory, M., Moser, B., & Freudenthaler, B. (2020) · ACM Conference on User Modeling, Adaptation and Personalization (UMAP) · doi:10.1145/3386392.3399562 · Cited by 11 · see citing works
Deep Learning architectures for generalized immunofluorescence based nuclear image segmentationPreprint
Kromp, F., Fischer, L., Bozsaky, E., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.12975 · Cited by 11 · see citing works
Self-citationALOHA: an architectural-aware framework for deep learning at the edge
Meloni, P., Loi, D., Deriu, G., Pimentel, A. D., Sapra, D., Moser, B., Shepeleva, N., Conti, F., Benini, L., Ripolles, O., et al. (2018) · Proceedings of the Workshop on INTelligent Embedded Systems Architectures and Applications · doi:10.1145/3285017.3285019 · Cited by 10 · see citing works
Automated Data Quality Monitoring.
Ehrlinger, L., & Wöß, W. (2017) · ICIQ · Cited by 10 · see citing works
Geist, V., Moser, M., Pichler, J., dos Santos, R. D. B., & Wieser, V. (2020) · Software Practice and Experience · doi:10.1002/spe.2933 · Cited by 9 · see citing works
la Prieta, F. D., Mathieu, P., Rincon, J. A., Bolock, A. E., del Val, E., Prunera, J. M. J., Carneiro, J., Fuentes{-}Fern{\\'{a}}ndez, R. (., Lopes, F., & Julian, V. (2020) · Communications in computer and information science · doi:10.1007/978-3-030-51999-5 · Cited by 9 · see citing works
Research on Interdisciplinary Characteristics: A Case Study in the Field of Artificial Intelligence
Li, S. L., & Wang, Y. (2019) · IOP Conference Series Materials Science and Engineering · doi:10.1088/1757-899x/677/5/052023 · Cited by 8 · see citing works
On generalization in moment-based domain adaptation
Zellinger, W., Moser, B. A., & Saminger-Platz, S. (2020) · Annals of Mathematics and Artificial Intelligence · doi:10.1007/s10472-020-09719-x · Cited by 8 · see citing works
Sobieczky, F. (2007) · Cited by 8 · see citing works
Explainability of AI-predictions based on psychological profiling
Neugebauer, S., Rippitsch, L., Sobieczky, F., & Geiß, M. (2021) · Procedia Computer Science · doi:10.1016/j.procs.2021.01.361 · Cited by 8 · see citing works
Linear Optimization Approach for Depth Range Adaption of Stereoscopic Videos
Zellinger, W., Moser, B., Chouikhi, A., Seitner, F. H., Nezveda, M., & Gelautz, M. (2016) · Electronic Imaging · doi:10.2352/issn.2470-1173.2016.5.sda-431 · Cited by 7 · see citing works
On Data Augmentation and Adversarial Risk: An Empirical AnalysisPreprint
Eghbal-zadeh, H., Koutini, K., Primus, P., Haunschmid, V., Lewandowski, M., Zellinger, W., Moser, B., & Widmer, G. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.02650 · Cited by 7 · see citing works
DaQL 2.0: Measure Data Quality based on Entity Models
Lettner, C., Stumptner, R., Fragner, W., Rauchenzauner, F., & Ehrlinger, L. (2021) · Procedia Computer Science · doi:10.1016/j.procs.2021.01.327 · Cited by 7 · see citing works
Sobieczky, F. (2010) · Journal of Theoretical Probability · doi:10.1007/s10959-010-0298-3 · Cited by 6 · see citing works
Eghbalzadeh, H., Fischer, L., Popitsch, N., Kromp, F., Taschner-Mandl, S., Gerber, T., Bozsaky, E., Ambros, P., Ambros, I., Widmer, G., et al. (2019) · J. Comput. Biol. · doi:10.1089/cmb.2018.0172 · Cited by 5 · see citing works
Moment-Based Domain Adaptation: Learning Bounds and AlgorithmsPreprint
Zellinger, W. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.10618 · Cited by 4 · see citing works
Variational Optimization of Informational Privacy
Kumar, M., Brunner, D. J., Moser, B., & Freudenthaler, B. (2020) · Communications in computer and information science · doi:10.1007/978-3-030-59028-4_4 · Cited by 4 · see citing works
Integration of Knowledge and Task Management in an Evolving, Communication-intensive Environment
Hübscher, G., Geist, V., Auer, D., Hübscher, N., & Küng, J. (2020) · International Conference on Information Integration and Web-based Applications & Services (iiWAS) · doi:10.1145/3428757.3429260 · Cited by 4 · see citing works
Predicting Adversarial Examples with High ConfidencePreprint
Galloway, A., Taylor, G. W., & Moussa, M. A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.04457 · Cited by 4 · see citing works
Bounds for the annealed return probability on large finite percolation graphs
Sobieczky, F. (2012) · Electronic Journal of Probability · doi:10.1214/ejp.v17-2329 · Cited by 3 · see citing works
Meloni, P., Loi, D., Deriu, G., Pimentel, A. D., Saprat, D., Pintor, M., Biggio, B., Ripollés, O., Solans, D., Conti, F., et al. (2018) · doi:10.1109/icm.2018.8704093 · Cited by 2 · see citing works
Tackling Semantic Shift in Industrial Streaming Data Over Time
Ehrlinger, L., Lettner, C., & Himmelbauer, J. (2020) · Databases, Knowledge, and Data Applications · Cited by 2 · see citing works
Flagged first, then by citations
Research works printed but not linked: 9
- Holzinger, A., Kieseberg, P., Tjoa, A.M., Weippl, E.R., Eds.; Springer: Berlin/Heidelberg, Germany, 2020; Volume 12279, pp. 451-471.
- Vapnik, V.N. Statistical Learning Theory; Wiley-Interscience: Hoboken, NJ, USA, 1998.
- Gal, Y. Uncertainty in Deep Learning. Ph.D. Thesis, University of Cambridge, Cambridge, UK, 2016.
- Chaudhuri, K., Salakhutdinov, R., Eds.; PMLR: Long Beach, CA, USA, 2019; Volume 97, pp. 6438-6447.
- Rigaki, M.; Garcia, S. A Survey of Privacy Attacks in Machine Learning. arXiv 2020, arXiv:2007.07646. · arXiv
- De La Prieta, F., Mathieu, P., Rincón Arango, J.A., El Bolock, A., Del Val, E., Jordán Prunera, J., Carneiro, J., Fuentes, R., Lopes, F., Julian, V., Eds.; Springer International Publishing: Cham, Switzerland, 2020; pp. 42-53.
- Kersting, K., Ed.; TU Darmstadt, Darmstadt, Germany, 2020; Volume 34, pp. 193-198.
- Navab, N.; Hornegger, J.; Wells, W.M.; Frangi, A.F., Eds.; Springer: Cham, Switzerland, 2015; pp. 234-241. 120.
- He, K.; Gkioxari, G.; Dollár, P.; Girshick, R. Mask R-CNN. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), Venice, Italy, 22-29 October 2017. 121.
Cited by
Sorted by citations
Steidl, M., Felderer, M., & Ramler, R. (2023) · Journal of Systems and Software · doi:10.1016/j.jss.2023.111615 · Cited by 126 · see citing works
Towards CRISP-ML(Q): A Machine Learning Process Model with Quality Assurance Methodology
Studer, S., Bui, T. B., Drescher, C., Hanuschkin, A., Winkler, L., Peters, S., & Müller, K. (2021) · Machine Learning and Knowledge Extraction · doi:10.3390/make3020020 · Cited by 127 · see citing works
Automation and artificial intelligence in filamentous fungi-based bioprocesses: A review
Wainaina, S., & Taherzadeh, M. J. (2022) · Bioresource Technology · doi:10.1016/j.biortech.2022.128421 · Cited by 70 · see citing works
Damar, M., Özen, A., Çakmak, Ü. E., Özoğuz, E., & Erenay, F. S. (2024) · Journal of AI · doi:10.61969/jai.1512906 · Cited by 47 · see citing works
Data Catalogs: A Systematic Literature Review and Guidelines to Implementation
Ehrlinger, L., Schrott, J., Melichar, M., Kirchmayr, N., & Wöß, W. (2021) · Communications in computer and information science · doi:10.1007/978-3-030-87101-7_15 · Cited by 32 · see citing works
mlpack 4: a fast, header-only C++ machine learning library
Curtin, R. R., Edel, M., Shrit, O., Agrawal, S., Basak, S., Balamuta, J., Birmingham, R., Dutt, K., Eddelbüttel, D., Garg, R., et al. (2023) · The Journal of Open Source Software · doi:10.21105/joss.05026 · Cited by 31 · see citing works
Trusted Artificial Intelligence: Towards Certification of Machine Learning ApplicationsPreprint
Winter, P. M., Eder, S. K., Weissenböck, J., Schwald, C., Doms, T., Vogt, T., Hochreiter, S., & Nessler, B. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.16910 · Cited by 34 · see citing works
Khuat, T. T., Kedziora, D. J., & Gabryś, B. (2023) · Foundations and Trends® in Human–Computer Interaction · doi:10.1561/1100000091 · Cited by 35 · see citing works
Nizamani, M. M., Zhang, H.-L., & Lai, Z. (2025) · Technology in Society · doi:10.1016/j.techsoc.2025.103121 · Cited by 25 · see citing works
Towards a Data Engineering Process in Data-Driven Systems Engineering
Petersen, P., Stage, H., Langner, J., Ries, L., Rigoll, P., Hohl, C. P., & Sax, E. (2022) · IEEE International Symposium on Systems Engineering · doi:10.1109/isse54508.2022.10005441 · Cited by 17 · see citing works
Biological mechanisms contradict AI consciousness: The spaces between the notes
Miller, W. B., Baluška, F., Reber, A. S., & Slijepčević, P. (2024) · Biosystems · doi:10.1016/j.biosystems.2024.105387 · Cited by 19 · see citing works
How Do Deep-Learning Framework Versions Affect the Reproducibility of Neural Network Models?
Shahriari, M., Ramler, R., & Fischer, L. (2022) · Machine Learning and Knowledge Extraction · doi:10.3390/make4040045 · Cited by 16 · see citing works
Approaches for the Use of AI in Workplace Health Promotion and Prevention: Systematic Scoping Review
Lange, M., Löwe, A., Kayser, I., & Schaller, A. (2024) · JMIR AI · doi:10.2196/53506 · Cited by 16 · see citing works
Park, S. J., Lee, J. Y., & Lee, J. (2024) · Data & Knowledge Engineering · doi:10.1016/j.datak.2023.102264 · Cited by 15 · see citing works
Navigation line detection algorithm for corn spraying robot based on improved LT-YOLOv10s
Diao, Z., Ma, S., Li, J., Zhang, J., Li, X., Zhao, S., He, Y. L., Zhang, B., & Jiang, L. (2025) · Precision Agriculture · doi:10.1007/s11119-025-10243-3 · Cited by 14 · see citing works
Bechara, H., Ibrahim, R., Zemouri, R. M., Kedjar, B., Merkhouf, A., Tahan, A. S., & Al‐Haddad, K. (2024) · IEEE Access · doi:10.1109/access.2024.3502546 · Cited by 13 · see citing works
What About the Data? A Mapping Study on Data Engineering for AI Systems
Heck, P. (2024) · IEEE/ACM International Conference on AI Engineering – Software Engineering for AI (CAIN) · doi:10.1145/3644815.3644954 · Cited by 12 · see citing works
The State of Documentation Practices of Third-Party Machine Learning Models and Datasets
Oreamuno, E. L., Khan, R. M., Bangash, A. A., Stinson, C., & Adams, B. (2024) · IEEE Software · doi:10.1109/ms.2024.3366111 · Cited by 12 · see citing works
Cherner, T. S., Foulger, T. S., & Donnelly, M. (2024) · TechTrends · doi:10.1007/s11528-024-01023-3 · Cited by 8 · see citing works
Homomorphic Encryption for AI-Based Applications: Challenges and Opportunities
Hamza, R. (2023) · International Conference on Knowledge and Systems Engineering/Proceedings · doi:10.1109/kse59128.2023.10299436 · Cited by 7 · see citing works
Context-based and image-based subsea pipeline degradation monitoring
Spahić, R., Lundteigen, M. A., & Hepsø, V. (2023) · Discover Artificial Intelligence · doi:10.1007/s44163-023-00063-7 · Cited by 7 · see citing works
Suryadevara, M., Rangineni, S., & Venkata, S. (2023) · International Journal of Science and Research (IJSR) · doi:10.21275/sr23719211528 · Cited by 6 · see citing works
Automation and Development Effort in Continuous AI Development: A Practitioners’ Survey
Steidl, M., Golendukhina, V., Felderer, M., & Ramler, R. (2023) · Euromicro Conference on Software Engineering and Advanced Applications (SEAA) · doi:10.1109/seaa60479.2023.00027 · Cited by 6 · see citing works
Enhancing DataOps practices through innovative collaborative models: A systematic review
Fannouch, A., Gharib, J., & Gahi, Y. (2025) · International Journal of Information Management Data Insights · doi:10.1016/j.jjimei.2025.100321 · Cited by 5 · see citing works
Digital Assistance Systems to Implement Machine Learning in Manufacturing: A Systematic Review
Rosemeyer, J., Pinzone, M., & Metternich, J. (2024) · Machine Learning and Knowledge Extraction · doi:10.3390/make6040134 · Cited by 5 · see citing works
Polato, M., Hammer, B., Röder, M., & Schleif, F. (2026) · Neurocomputing · doi:10.1016/j.neucom.2026.132671 · Cited by 6 · see citing works
Automating Data Quality Monitoring with Reference Data Profiles
Ehrlinger, L., Werth, B., & Wöß, W. (2023) · Communications in computer and information science · doi:10.1007/978-3-031-37890-4_2 · Cited by 4 · see citing works
Lin, C.-T., & Huang, S.-J. (2024) · Information and Software Technology · doi:10.1016/j.infsof.2024.107449 · Cited by 8 · see citing works
Responsible AI Adoption Through Private-Sector Governance
Wiesmüller, S., Fischer, N., Mehnert, W., & Ammon, S. (2023) · CSR, sustainability, ethics & governance · doi:10.1007/978-3-031-09245-9_5 · Cited by 4 · see citing works
Large Language Model-Based Autonomous Agent for Prognostics and Health Management
Cha, M., Yoon, S.-I., Kim, S., Kang, D., Nam, K., Lee, T., & Kim, J.-Y. (2025) · Machines · doi:10.3390/machines13090831 · Cited by 5 · see citing works
Git Workflow for Active Learning: A Development Methodology Proposal for Data-Centric AI Projects
Stieler, F., & Bauer, B. (2023) · International Conference on Evaluation of Novel Approaches to Software Engineering · doi:10.5220/0011988400003464 · Cited by 3 · see citing works
Artificial intelligence for model-based systems engineering: A scoping review
Song, Z., Tan, Y., Jin, C., Xu, Q., & Zheng, X. (2026) · Computers & Industrial Engineering · doi:10.1016/j.cie.2026.112178 · Cited by 2 · see citing works
Como gerenciar os riscos de inteligência artificial
Brandão, C. E., & Becker, J. L. (2022) · GV executivo \b (Impresso)/GV Executivo · doi:10.12660/gvexec.v21n4.2022.88403 · Cited by 2 · see citing works
Fuchs, M., Fischer, L., Montuoro, A., Kumar, M., & Moser, B. (2024) · Communications in computer and information science · doi:10.1007/978-3-031-68302-2_6 · Cited by 2 · see citing works
Data Integration, Management, and Quality: From Basic Research to Industrial Application
Ehrlinger, L., Lettner, C., Fragner, W., Gsellmann, G., Nestelberger, S., Rauchenzauner, F., Schützeneder, S., Tiefengrabner, M., & Zeindl, J. (2022) · Communications in computer and information science · doi:10.1007/978-3-031-14343-4_16 · Cited by 2 · see citing works
Fox, S., & Rey, V. F. (2024) · Machine Learning and Knowledge Extraction · doi:10.3390/make6010027 · Cited by 2 · see citing works
Improving Safety of AI Systems
Suryanarayana, S., Sangwan, R. S., Srinivasan, S., & Badr, Y. (2024) · IEEE reliability magazine · doi:10.1109/mrl.2024.3473848 · Cited by 1 · see citing works
Towards Practicable Machine Learning Development Using AI Engineering Blueprints
Weeger, N., Stiehl, A., von Kistowski, J., Geißelsöder, S., & Uhl, C. (2025) · 2025 IEEE 22nd International Conference on Software Architecture Companion (ICSA-C) · doi:10.1109/icsa-c65153.2025.00078 · Cited by 2 · see citing works
Regulatory compliance and ethical considerations in integration artificial intelligence
Neguț, I., & Vișan, A. I. (2025) · Elsevier eBooks · doi:10.1016/b978-0-443-34076-5.00025-0 · Cited by 2 · see citing works
AI and ML in Understanding Biosphere and Cross-Sphere Interactions
Patra, P. P., & Behera, S. (2026) · Earth and environmental sciences library · doi:10.1007/978-3-032-11426-6_14 · Cited by 1 · see citing works
Efficient Wild Animal Detection and Collection Using Quantized Models on Low-End Edge Devices
Huang, X., Mirri, S., Shen, L. H., & Tang, S.-K. (2025) · IEEE Symposium on Computers and Communications (ISCC) · doi:10.1109/iscc65549.2025.11325936 · Cited by 1 · see citing works
Supervised Machine Learning for Technical Debt in Python: Analysis and Prediction
Fırıncı, E., & Alayedi, M. (2026) · Machine Learning and Knowledge Extraction · doi:10.3390/make8050118 · Cited by 1 · see citing works
Software Engineering for AI-Driven Building Operation
Zech, P., Hammes, S., Weninger, J., Pannosch, J., & Steindl, G. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-36590-3_7 · Cited by 1 · see citing works
Roadmap Analysis of Artificial Intelligence Engineering Method
Sandfreni, S., & Budiardjo, E. K. (2024) · Revue d intelligence artificielle · doi:10.18280/ria.380305 · Cited by 1 · see citing works
Make or buy strategy for Machine Learning Operations – MLOps
Nogare, D., Silveira, I. F., BANZAI, R., & ALEXANDRE, M. C. (2025) · Anais da Academia Brasileira de Ciências · doi:10.1590/0001-3765202520240924 · Cited by 1 · see citing works
Collaborative Aspects of Solving Rail-Track Multi-sensor Data Fusion
Kromp, F., Hinterberger, F., Konanur, D., & Wieser, V. (2022) · Communications in computer and information science · doi:10.1007/978-3-031-14343-4_7 · Cited by 1 · see citing works
Special Issue “Selected Papers from CD-MAKE 2020 and ARES 2020”
Weippl, E. R., Holzinger, A., & Kieseberg, P. (2023) · Machine Learning and Knowledge Extraction · doi:10.3390/make5010012 · Cited by 1 · see citing works
Huang, S.-J., & Lin, C.-T. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4511875
Examples of Long-Term Science–Industry Partnerships for Translational Computer Science
Travnicek, C., Nowak, V., Ramler, R., Fischer, L., & Bühler, K. (2023) · Computing in Science & Engineering · doi:10.1109/mcse.2023.3286249
Introduction to Artificial Intelligence
Nagaraj, A. (2023) · BENTHAM SCIENCE PUBLISHERS eBooks · doi:10.2174/9789815165708123010005
Knowledge Guided Clustering Medieval Polychromy
Sobieczky, F., & Sobieczky, E. (2024) · Communications in computer and information science · doi:10.1007/978-3-031-68302-2_10
Meijer, W. (2024) · ACM/IEEE International Conference on Model Driven Engineering Languages and Systems (MODELS) · doi:10.1145/3652620.3688201
Towards a Design Taxonomy of AI Products and Services
Norris, S., Resmini, A., Lund, J., & Lindberg, S. (2025) · Communications in computer and information science · doi:10.1007/978-3-031-94171-9_37
Responsible AI in Organizational Decision-Making: A Conceptual Framework
Salamzadeh, Y. (2025) · Contributions to management science · doi:10.1007/978-3-032-02526-5_4
Kurcjusz, M., & Das, R. R. (2025) · Acta Scientiarum Polonorum Architectura · doi:10.22630/aspa.2025.24.28
Tariq, M. U., Afendi, A. S., Ismail, S. B., & Ramli, A. A. (2026) · Journal of Informatics and Web Engineering · doi:10.33093/jiwe.2026.5.1.23
Nogare, D., Silveira, I. F., & da Silva, L. A. (2025) · Congresso Latino-Americano de Software Livre e Tecnologias Abertas (Latinoware) · doi:10.5753/latinoware.2025.16455
System and Software Quality Requirements of AI Systems
Uysal, M. P. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/403119
Pourrahmani, H. (2026) · Elsevier eBooks · doi:10.1016/b978-0-443-45526-1.00006-4
Lisaru, C.-V., & Kifor, C. V. (2026) · Acta Informatica Pragensia · doi:10.18267/j.aip.303
Understanding AI: A Deep Dive into Its Forms, Functions, and Ethical Frontiers
Moloi, T. (2026) · Springer eBooks · doi:10.1007/978-981-95-8212-9_3
Generative AI in Systems Engineering: A Framework for Risk Assessment of Large Language Models
Otten, S., Reis, P., Rigoll, P., Schürmann, T., Ransiek, J., Langner, J., & Sax, E. (2026) · IEEE International Systems Conference (SysCon) · doi:10.1109/syscon66367.2026.11503606
Integration of AI in Data Requirements for Stuttering-Aware Speech Recognition Systems
Altraide, I., & Simske, S. (2026) · Conference on systems engineering research series · doi:10.1007/978-3-032-12309-1_33
Lakkaraju, K., Valluru, S. L., Srivastava, B., & Valtorta, M. (2026) · International World Wide Web Conference (WWW) · doi:10.1145/3774905.3793129
Kuşçu, A. İ., Çınarer, G., & Kuşçu, S. (2026) · Anatolian Current Medical Journal · doi:10.38053/acmj.1848512
Human and Machine Synergy in Agriculture and Food Systems
М., К., & Seethapathy, P. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-7036-1.ch008
Agin, F., Fauteux‐Lefebvre, C., & Thibault, J. (2026) · The Canadian Journal of Chemical Engineering · doi:10.1002/cjce.70483
Interfacing Minds, Organizing Interactions: A Novel Human-AI Interaction Framework for Dyadic Teams
Merwin, E. R., Brennan, L., Gonzalez, G. M., Lazzara, E. H., & Keebler, J. R. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-30846-7_22
Statistical Approaches vs. Artificial Intelligence Models in Scientific Inquiry: What to Choose When
Wahi, M. M., Poluboiartsev, I., & Chacko, D. (2026) · AI Computer Science and Robotics Technology · doi:10.5772/acrt.20250263
Bier, A., Osiński, K., Wojtkowska, M., & Jastrzębska, A. (2026) · Information and Software Technology · doi:10.1016/j.infsof.2026.108345
Cloud-Based AI Solutions for Ensuring Data Quality in Predictive Models
Saleh, S. M., Gunda, V., Hardt, W., & Hasan, R. (2026) · International Symposium on Computer Science and Educational Technology (ISCSET) · doi:10.1109/iscset70652.2026.11722978
- Authors
- Fischer, L., Ehrlinger, L., Geist, V., Ramler, R., Sobiezky, F., Zellinger, W., Brunner, D., Kumar, M., & Moser, B.
- Affiliations
- Software Competence Center Hagenberg (Austria) (AT)
- Johannes Kepler University of Linz (AT)
- Software Competence Center Hagenberg GmbH (SCCH), 4232 Hagenberg, Austria
- Corresponding author
- Fischer, Lukas
- ORCID
- Venue
- Machine Learning and Knowledge Extraction
- Publisher
- MDPI AG
- Published
- 2020-12-31
- Volume
- 3
- Issue
- 1
- Pages
- 56-83
- ISSN
- 2504-4990 (ISSN-L)
- OpenAlex
- W3117442988
- Cited by
- 71
- References
- 163
- Citations by year
- 2026: 20 · 2025: 12 · 2024: 16 · 2023: 14 · 2022: 6 · 2021: 3
- FWCI
- 5.41
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Machine Learning and Knowledge Extraction (publishedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Bundesministerium für Klimaschutz, Umwelt, Energie, Mobilität, Innovation und Technologie
- Österreichische Forschungsförderungsgesellschaft
- Bundesministerium für Digitalisierung und Wirtschaftsstandort
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence Applications
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Keywords
- machine learning systems, deep learning models, data quality assurance, software quality, human-centered AI, AI ethics, confidentiality, model deployment, theory-practice gap
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review