Software engineering for machine-learning applications: The road ahead
Abstract
The First Symposium on Software Engineering for Machine Learning Applications (SEMLA) aimed to create a space in which machine learning (ML) and software engineering (SE) experts could come together to discuss challenges, new insights, and practical ideas regarding the engineering of ML and AI-based systems. Key challenges discussed included the accuracy of systems built using ML and AI models, the testing of those systems, industrial applications of AI, and the rift between the ML and SE communities. This article is part of a theme issue on software engineering’s 50th anniversary.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Existing machine learning-based firmware defect detection methods can be divided into three categories based on context-free function matching and template configuration defect detection.
“The existing machine learning-based firmware defect detection methods can be divided into three categories based on context-free function matching and template configuration defect detection [44].”
From A Survey of IoT Privacy Security: Architecture, Technology, Challenges, and Trends (2024) · cites this paper as [44]
The difficulties of debugging ML systems were pointed out by several studies.
“The difficulties of debugging ML systems were pointed out by several studies (Khomh et al., 2018;Wan et al., 2019;Zhang et al., 2018b;Zhang et al., 2019a).”
From A software engineering perspective on engineering machine learning systems: State of the art and challenges (2021) · cites this paper as Khomh et al., 2018
Unlike traditional software, code coverage cannot be used for ML components since the decision logic of ML capability is obtained via training an ML model, not via explicit coding.
“Unlike traditional software, code coverage cannot be used for ML components since the decision logic of ML capability is obtained via training an ML model, not via explicit coding (Ishikawa and Yoshioka, 2019;Khomh et al., 2018;Kim et al., 2019;Ma et al., 2018b;Sekhon and Fleming, 2019;Sun et al., 2018;Zhang et al., 2020c).”
From A software engineering perspective on engineering machine learning systems: State of the art and challenges (2021) · cites this paper as Khomh et al., 2018
The size of test case sets is enormous, and missing test cases is a more challenging problem for ML systems.
“The size of test case sets is enormous, and missing test cases is a more challenging problem for ML systems (Khomh et al., 2018).”
From A software engineering perspective on engineering machine learning systems: State of the art and challenges (2021) · cites this paper as Khomh et al., 2018
Cited by
Sorted by citations
Requirements Engineering for Machine Learning: Perspectives from Data Scientists
Vogelsang, A., & Borg, M. (2019) · IEEE International Requirements Engineering Conference Workshops (REW) · doi:10.1109/rew.2019.00050 · Cited by 238 · see citing works
Giray, G. (2021) · Journal of Systems and Software · doi:10.1016/j.jss.2021.111031 · Cited by 187 · see citing works
A Survey of IoT Privacy Security: Architecture, Technology, Challenges, and Trends
Sun, P., Shen, S., Wan, Y., Wu, Z., Fang, Z., & Gao, X.-Z. (2024) · IEEE Internet of Things Journal · doi:10.1109/jiot.2024.3372518 · Cited by 172 · see citing works
“In addition to searching for security defects according to different context semantics in binary code, the method further enables analysts to check search results through the generated interpreter diagnostic report to determine the defect code displayed in the target program [44], [47].”
MethodRequirements engineering for artificial intelligence systems: A systematic mapping study
Ahmad, K., Abdelrazek, M., Arora, C., Bano, M., & Grundy, J. (2023) · Information and Software Technology · doi:10.1016/j.infsof.2023.107176 · Cited by 132 · see citing works
Adoption and Effects of Software Engineering Best Practices in Machine Learning
Serban, A., van der Blom, K., Hoos, H., & Visser, J. (2020) · Proceedings of the 14th ACM / IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM) · doi:10.1145/3382494.3410681 · Cited by 127 · see citing works
Understanding Development Process of Machine Learning Systems: Challenges and Solutions
Nascimento, E. D. S., Ahmed, I., Oliveira, E., Palheta, M., Steinmacher, I., & Conte, T. (2019) · International Symposium on Empirical Software Engineering and Measurement · doi:10.1109/esem.2019.8870157 · Cited by 72 · see citing works
Imran, M., Zaman, U., Imran, I., Imtiaz, J., Fayaz, M., & Gwak, J. (2021) · Electronics · doi:10.3390/electronics10202501 · Cited by 80 · see citing works
A Survey of the Selenium Ecosystem
García, B., Gallego, M., Gortázar, F., & Muñoz-Organero, M. (2020) · Electronics · doi:10.3390/electronics9071067 · Cited by 67 · see citing works
Zhou, Q., Zheng, K., Zhang, K., Hou, L., & Wang, X. (2022) · IEEE Internet of Things Journal · doi:10.1109/jiot.2022.3196269 · Cited by 64 · see citing works
“More and more researchers are interested in applying ML technologies to solve the problems of SE [36].”
BackgroundMachine Learning for Detecting Data Exfiltration
Sabir, B., Ullah, F., Babar, M. A., & Gaire, R. (2021) · ACM Computing Surveys · doi:10.1145/3442181 · Cited by 61 · see citing works
An Empirical Study of the Impact of Data Splitting Decisions on the Performance of AIOps Solutions
Lyu, Y., Li, H., Sayagh, M., Jiang, Z. M., & Hassan, A. E. (2021) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3447876 · Cited by 60 · see citing works
Software Engineering for AI-Based Systems: A Survey
Martínez‐Fernández, S., Bogner, J., Franch, X., Oriol, M., Siebert, J., Trendowicz, A., Vollmer, A. M., & Wagner, S. (2022) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3487043 · Cited by 374 · see citing works
What’s up with Requirements Engineering for Artificial Intelligence Systems?
Ahmad, K., Bano, M., Abdelrazek, M., Arora, C., & Grundy, J. (2021) · IEEE International Requirements Engineering Conference (RE) · doi:10.1109/re51729.2021.00008 · Cited by 61 · see citing works
Requirements practices and gaps when engineering human-centered Artificial Intelligence systems
Ahmad, K., Abdelrazek, M., Arora, C., Bano, M., & Grundy, J. (2023) · Applied Soft Computing · doi:10.1016/j.asoc.2023.110421 · Cited by 57 · see citing works
Software Architecture Challenges for ML Systems
Lewis, G. A., Özkaya, İ., & Xu, X. (2021) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsme52107.2021.00071 · Cited by 57 · see citing works
Deep-Learning-Incorporated Augmented Reality Application for Engineering Lab Training
Estrada, J., Paheding, S., Yang, X., et al. (2022) · Applied Sciences · doi:10.3390/app12105159 · Cited by 44 · see citing works
Moreb, M., Mohammed, T. A., & Bayat, O. (2020) · IEEE Access · doi:10.1109/access.2020.2970178 · Cited by 39 · see citing works
Habiba, U.-E., Haug, M., Bogner, J., & Wagner, S. (2024) · Requirements Engineering · doi:10.1007/s00766-024-00432-3 · Cited by 51 · see citing works
“Despite substantial growth in AI-based systems, we continue to see significant challenges and project failures [2].”
Risk-based data validation in machine learning-based software systems
Foidl, H., & Felderer, M. (2019) · doi:10.1145/3340482.3342743 · Cited by 38 · see citing works
“However, the quality assurance of ML-based software systems is a very challenging task [8, 9] and currently discussed as a hot and important research topic [19].”
BackgroundRabe, M., Milz, S., & Mäder, P. (2021) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw53098.2021.00023 · Cited by 36 · see citing works
Nakamichi, K., Ohashi, K., Namba, I., et al. (2020) · IEEE International Requirements Engineering Conference (RE) · doi:10.1109/re48521.2020.00036 · Cited by 33 · see citing works
Model driven engineering for machine learning components: A systematic literature review
Naveed, H., Arora, C., Khalajzadeh, H., Grundy, J., & Haggag, O. (2024) · Information and Software Technology · doi:10.1016/j.infsof.2024.107423 · Cited by 38 · see citing works
Architectural Design Decisions for the Machine Learning Workflow
Warnett, S. J., & Zdun, U. (2022) · Computer · doi:10.1109/mc.2021.3134800 · Cited by 32 · see citing works
Czub, N., Szlęk, J., Pacławski, A., et al. (2023) · Molecular Pharmaceutics · doi:10.1021/acs.molpharmaceut.2c01117 · Cited by 27 · see citing works
“32 Being applied on a homogeneous dataset of serotonergic drugs, our approach can grant better correlation with actual HIA, so as to promote the development of useful tools for effective early screening of oral drug candidates.”
Software engineering practices for machine learning — Adoption, effects, and team assessment
Serban, A., van der Blom, K., Hoos, H. H., & Visser, J. (2023) · Journal of Systems and Software · doi:10.1016/j.jss.2023.111907 · Cited by 29 · see citing works
The Machine Learning Life Cycle in Chemical Operations – Status and Open Challenges
Gärtler, M., Khaydarov, V., Klöpper, B., & Urbas, L. (2021) · Chemie Ingenieur Technik · doi:10.1002/cite.202100134 · Cited by 25 · see citing works
A Systematic Mapping of Quality Models for AI Systems, Software and Components
Ali, M. A., Yap, N. K., Ghani, A., Zulzalil, H. B., Admodisastro, N. I., & Najafabadi, A. A. (2022) · Applied Sciences · doi:10.3390/app12178700 · Cited by 24 · see citing works
Machine learning experiment management tools: a mixed-methods empirical study
Idowu, S., Osman, O., Strüber, D., et al. (2024) · Empirical Software Engineering · doi:10.1007/s10664-024-10444-w · Cited by 17 · see citing works
“The user-friendly interfaces and interactive visualizations these tools provide simplify exploring and retrieving experiment assets.”
Is Machine Learning Software Just Software: A Maintainability View
Mikkonen, T., Nurminen, J. K., Raatikainen, M., Fronza, I., Mäkitalo, N., & Männistö, T. (2021) · Lecture notes in business information processing · doi:10.1007/978-3-030-65854-0_8 · Cited by 21 · see citing works
An MDE Method for Improving Deep Learning Dataset Requirements Engineering using Alloy and UML
Ries, B., Guelfi, N., & Jahić, B. (2021) · International Conference on Model-Driven Engineering and Software Development · doi:10.5220/0010216600410052 · Cited by 21 · see citing works
Gong, L., Zhang, J., Wei, M., Zhang, H., & Huang, Z. (2022) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3569934 · Cited by 16 · see citing works
Green-IN Machine Learning at a Glance
Gutiérrez, M. E., Moraga, M. Á., García, F. O., & Calero, C. (2023) · Computer · doi:10.1109/mc.2023.3254646 · Cited by 16 · see citing works
“…other AI approaches, such as neural networks, as well as for the integration of ML or AI lifecycles into the general software development lifecycle, 16 thus obtaining a complete cycle for AI-based software applications and taking full advantage of the benefits offered by the green-in software…”
Sensemaking Practices in the Everyday Work of AI/ML Software Engineering
Wolf, C. T., & Paine, D. (2020) · doi:10.1145/3387940.3391496 · Cited by 16 · see citing works
“…in AI/ML work (e.g., accuracy and reproducibility of AI/ML model outputs cannot be known beforehand), comprehensibility of complex model functions and behavior (e.g., AI explainability), and managing expectations with clients (e.g., idealistic or hyped expectations of what is possible) [2, 10, 12].”
BackgroundMaking Sense of AI Systems Development
Dolata, M., & Crowston, K. (2023) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2023.3338857 · Cited by 15 · see citing works
“Second, it advances the conversation about AI’s impact on SE processes [4], [23], [24] by focusing on collaborative, inter-organizational projects and the socio-cognitive aspects of software development.”
InfluentialRaffin, T., Reichenstein, T., Klier, D., Kühl, A., & Franke, J. E. (2022) · Procedia CIRP · doi:10.1016/j.procir.2022.10.063 · Cited by 14 · see citing works
A Requirements Engineering Perspective to AI-Based Systems Development: A Vision Paper
Franch, X., Jedlitschka, A., & Martínez‐Fernández, S. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-29786-1_15 · Cited by 14 · see citing works
Machine Learning for Detecting Data Exfiltration: A ReviewPreprint
Sabir, B., Ullah, F., Babar, M. A., & Gaire, R. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.09344 · Cited by 13 · see citing works
Systematic Mapping Study on the Machine Learning Lifecycle
Xie, Y., Cruz, L., Heck, P., & Rellermeyer, J. S. (2021) · 2021 IEEE/ACM 1st Workshop on AI Engineering - Software Engineering for AI (WAIN) · doi:10.1109/wain52551.2021.00017 · Cited by 14 · see citing works
State-of-the-Art and Challenges of Engineering ML- Enabled Software Systems in the Deep Learning Era
Assres, G. M., Bhandari, G. P., Shalaginov, A., Grønli, T., & Ghinea, G. (2025) · ACM Computing Surveys · doi:10.1145/3731597 · Cited by 15 · see citing works
“Consequently, software engineering researchers are delving into the potential of ML to craft intelligent systems, automating routine tasks, optimizing intricate processes, and enhancing overall system performance.”
On the Impact of Multi-language Development in Machine Learning Frameworks
Grichi, M., Eghan, E. E., & Adams, B. (2020) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsme46990.2020.00058 · Cited by 15 · see citing works
Self-citationCharacterization and Automatic Updates of Deprecated Machine-Learning API Usages
Haryono, S. A., Thung, F., Lo, D., et al. (2021) · IEEE International Conference on Software Maintenance and Evolution (ICSME) · doi:10.1109/icsme52107.2021.00019 · Cited by 11 · see citing works
Machine learning application development: practitioners’ insights
Rahman, M. S., Khomh, F., Hamidi, A., et al. (2023) · Software Quality Journal · doi:10.1007/s11219-023-09621-9 · Cited by 11 · see citing works
Self-citationLorenzoni, G., Alencar, P. S. C., do Nascimento, N. M., & Cowan, D. D. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.07574 · Cited by 20 · see citing works
Integrated multi-view modeling for reliable machine learning-intensive software engineering
Husen, J. H., Washizaki, H., Runpakprakun, J., Yoshioka, N., Tun, H. T., Fukazawa, Y., & Takeuchi, H. (2024) · Software Quality Journal · doi:10.1007/s11219-024-09687-z · Cited by 9 · see citing works
“Due to the criticality of the results, ML-based software must employ a reliable and trustworthy approach (Khomh et al., 2018; Ozkaya, 2020).”
Álvarez-Rodríguez, J. M., Zuñiga, R. M., Moreno, V., et al. (2019) · INCOSE International Symposium · doi:10.1002/j.2334-5837.2019.00621.x · Cited by 9 · see citing works
“In this context, the unavoidable convergence of the Systems & Software Engineering and Artificial Intelligence and Machine Learning disciplines is considered a must (Khomh et al. 2018) and one of the next major challenges within the engineering process.”
BackgroundAmézquita, L., Castillo, O., Soria, J., et al. (2021) · Studies in computational intelligence · doi:10.1007/978-3-030-68776-2_2 · Cited by 9 · see citing works
A Reference Architecture for Applications with Conversational Components
Srivastava, S., & Prabhakar, T. V. (2019) · IEEE International Conference on Software Engineering and Service Science (ICSESS) · doi:10.1109/icsess47205.2019.9040822 · Cited by 8 · see citing works
Human-centric Requirements Engineering for Artificial Intelligence Software Systems
Ahmad, K. (2021) · IEEE International Requirements Engineering Conference (RE) · doi:10.1109/re51729.2021.00070 · Cited by 8 · see citing works
Cross Fertilization in Software Engineering
Colomo‐Palacios, R. (2020) · Communications in computer and information science · doi:10.1007/978-3-030-56441-4_1 · Cited by 8 · see citing works
Engineering Reliable Deep Learning SystemsPreprint
Santhanam, P., Farchi, E., & Pankratius, V. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.12582 · Cited by 17 · see citing works
Requirements Elicitation and Modelling of Artificial Intelligence Systems: An Empirical Study
Ahmad, K., Abdelrazek, M., Arora, C., Grundy, J., & Bano, M. (2023) · International Conference on Evaluation of Novel Approaches to Software Engineering · doi:10.5220/0011842300003464 · Cited by 10 · see citing works
Preliminary Systematic Literature Review of Machine Learning System Development Process
Watanabe, Y., Washizaki, H., Sakamoto, K., Saito, D., Honda, K., Tsuda, N., Fukazawa, Y., & Yoshioka, N. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.05528 · Cited by 15 · see citing works
Trending Topics in Software Engineering
Forti, S., Breitenbücher, U., & Soldani, J. (2022) · ACM SIGSOFT Software Engineering Notes · doi:10.1145/3539814.3539820 · Cited by 7 · see citing works
Artificial Intelligence For Software Testing-Perspectives And Practices
Jha, N., & Popli, R. (2021) · International Conference on Computational Intelligence and Communication Technologies (CCICT) · doi:10.1109/ccict53244.2021.00075 · Cited by 7 · see citing works
Subbiah, U., Ramachandran, M., & Mahmood, Z. (2020) · Computer communications and networks · doi:10.1007/978-3-030-33624-0_7 · Cited by 6 · see citing works
Mahmudova, S. (2019) · International Conference on Technics, Technologies and Education ICTTE ... · doi:10.15547/ictte.2019.04.061 · Cited by 5 · see citing works
Toward a Method Engineering Framework for Project Management and Machine Learning
Uysal, M. P. (2023) · IEEE Annual International Computer Software and Applications Conference (COMPSAC) · doi:10.1109/compsac57700.2023.00179 · Cited by 4 · see citing works
Colomo‐Palacios, R. (2019) · Springer proceedings in complexity · doi:10.1007/978-3-030-30809-4_31 · Cited by 4 · see citing works
Metrics for Measuring Error Extents of Machine Learning Classifiers
Zhu, H., Bayley, I., & Green, M. A. (2022) · IEEE International Conference on Artificial Intelligence Testing (AITest) · doi:10.1109/aitest55621.2022.00016 · Cited by 4 · see citing works
Tablada-Dominguez, A., Muñoz, M., Ocharán-Hernández, J. O., & Sánchez-García, Á. J. (2023) · Mexican International Conference on Computer Science · doi:10.1109/enc60556.2023.10508685 · Cited by 3 · see citing works
Pintz, M., Becker, D. S., & Mock, M. (2024) · doi:10.1145/3643691.3648585 · Cited by 3 · see citing works
“Furthermore, research is being conducted into how these requirements can be operationalized through AI software engineering [26, 29], measures to safeguard AI systems [23] or through procedural requirements for stakeholders to implement Trustworthy AI-systems [30, 41].”
BackgroundA Systemic Approach to Machine Learning Project Management
Uysal, M. P., & Akturk, E. (2024) · IEEE Engineering Management Review · doi:10.1109/emr.2024.3503677 · Cited by 3 · see citing works
“(cid:2) ML developers:They concentrate on ML pipeline processes,data management,and ensure the quality and efficiency of the ML development and deployment environment.”
What Machine Learning Can Learn From Software Modularity
Kriens, P., & Verbelen, T. (2022) · Computer · doi:10.48550/arxiv.1906.10366 · Cited by 15 · see citing works
Re-orienting Toward the Science of the Artificial: Engineering AI Systems
Russell, S., Jalaian, B. A., & Moskowitz, I. S. (2021) · Springer eBooks · doi:10.1007/978-3-030-77283-3_8 · Cited by 3 · see citing works
Sharma, A. B., Prana, G. A. A., Sawhney, A., et al. (2022) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner53432.2022.00129 · Cited by 2 · see citing works
Review of ways to apply machine learning methods in software engineering
Asaad, J., & Avksentieva, E. Y. (2023) · E3S Web of Conferences · doi:10.1051/e3sconf/202344907018 · Cited by 2 · see citing works
Liywalii, E. (2024) · Communications in computer and information science · doi:10.1007/978-3-031-78255-8_25 · Cited by 2 · see citing works
Perera, P., Perera, S., Jin, X., Rashidi, M., Nanayakkara, S., Yazbek, G., & Yazbek, A. (2025) · Journal of Information Technology in Construction · doi:10.36680/j.itcon.2025.041 · Cited by 2 · see citing works
MLOps with Microservices: A Case Study on the Maritime Domain
Ferreira, R. C., Trapmann, R., & van den Heuvel, W. (2025) · Communications in computer and information science · doi:10.1007/978-3-032-07313-6_1 · Cited by 2 · see citing works
“The novel field of Software Engineering for Artificial Intelligence (SE4AI) studies the adaptation of Software Engineering techniques for the creation of ML-Enabled Systems (MLES) [9].”
Internet of Things and Artificial Intelligence—A Wining Partnership?
Semião, J., Santos, M. B., Teixeira, I. C., & Teixeira, J. P. (2020) · Internet of things · doi:10.1007/978-3-030-44907-0_15 · Cited by 2 · see citing works
Software Engineering for Data Mining (ML-Enabled) Software Applications
Saeed, S., Abubakar, M. M., & Karabatak, M. (2021) · International Symposium on Digital Forensics and Security (ISDFS) · doi:10.1109/isdfs52919.2021.9486319 · Cited by 2 · see citing works
An Initial Analysis of Repair and Side-effect Prediction for Neural Networks
Ishimoto, Y., Matsui, K., Kondo, M., et al. (2023) · IEEE/ACM International Conference on AI Engineering – Software Engineering for AI (CAIN) · doi:10.1109/cain58948.2023.00017 · Cited by 2 · see citing works
“Hence, the software engineering research community has started to study the quality assurance of neural network models as part of Software Engineering for AI (SE4AI) [3], [10].”
MethodAI Engineering for Safety-Critical Control Systems: An Aerospace Perspective
Durak, U., Daw, Z., Topcu, U., Atkins, E. M., Cofer, D. D., Uzun, M., Inalhan, G., Kosmidis, L., Paunicka, J., & Pham, T. T. (2025) · American Control Conference · doi:10.23919/acc63710.2025.11107793 · Cited by 1 · see citing works
“While the behavior of software-intensive systems is specified deductively, at the lowest level with code, the behavior of AI-based systems is derived from training data which can be regarded as an inductive specification of the requirements [3].”
Software Patterns to Identify Credit Risk Patterns
Bhatore, S., Reddy, Y. R., Sanagavarapu, L. M., et al. (2020) · IEEE India Council International Conference (INDICON) · doi:10.1109/indicon49873.2020.9342579 · Cited by 1 · see citing works
Towards Requirements Engineering Activities for Machine Learning-Enabled FinTech Applications
Li, Y., Keung, J., Bennin, K. E., et al. (2023) · Proceedings - Asia Pacific Software Engineering Conference/Proceedings · doi:10.1109/apsec60848.2023.00022 · Cited by 1 · see citing works
“ML was successfully and widely applied in image recognition, natural language processing, and outperforming humans in complex games [1, 2].”
MethodUysal, M. P. (2025) · Turkish Journal of Engineering · doi:10.31127/tuje.1527734 · Cited by 2 · see citing works
Systematic Mapping Study on the Machine Learning LifecyclePreprint
Xie, Y., Cruz, L., Heck, P., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.10248 · Cited by 1 · see citing works
An Intelligent Journey to Machine Learning Applications in Component-Based Software Engineering
Wangoo, D. P. (2020) · Algorithms for intelligent systems · doi:10.1007/978-981-15-0222-4_16
Search Based Risk Reduction Supporting the Intelligent Components Selection Process
Valenzuela-Toledo, P., Cares, C., & Diéguez, M. (2019) · IEEE CHILEAN Conference on Electrical, Electronics Engineering, Information and Communication Technologies (CHILECON) · doi:10.1109/chilecon47746.2019.8987553
Una mirada a la sincronización de la radio cognitiva
Sarmiento, D. A. L., & Viveros, L. J. H. (2020) · Inclusión & Desarrollo · doi:10.26620/uniminuto.inclusion.7.2.2020.50-62
Color Teams for Machine Learning DevelopmentPreprint
Kalin, J. D., Noever, D. A., & Ciolino, M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.10601
Trusting Human–Machine Teaming
Michael, J. B. (2021) · Computer · doi:10.1109/mc.2021.3106416
(2022) · Computer · doi:10.1109/mc.2022.3195369
Shanmugapriya, S., & P.Devika, M. (2023) · International Journal of Engineering Technology and Management Sciences · doi:10.46647/ijetms.2023.v07i02.077
Nokovic, B. (2023) · doi:10.5121/csit.2023.131206
“Software systems are traditionally constructed deductively by program code as the rules that govern the system behaviors, but with ML techniques those rules can be changed inductively from a set of trained data [7].”
BackgroundThe Role of Machine Learning in Software Development
Tawar, D. B. S. (2024) · International Journal of Innovative Science and Research Technology (IJISRT) · doi:10.38124/ijisrt/ijisrt24may2519
“[2].”
Requirements Elicitation in the Age of AI: A Tool’s Multi-system Journey
Ahmad, K., Arora, C., Abdelrazek, M., Grundy, J., & Vasa, R. (2024) · Communications in computer and information science · doi:10.1007/978-3-031-64182-4_4
Zhu, P., Lv, P., & Zhang, Y. (2025) · Journal of Future Artificial Intelligence and Technologies · doi:10.62411/faith.3048-3719-250
Exploring the Impact of GitHub Actions on Pull Request Reviews in Machine Learning Projects
Bernardo, J. H., da Costa, D. A., de Medeiros, S. Q., et al. (2025) · doi:10.5753/sbes.2025.10766
“This ,wherePRreviewsmustconsidernotonlycode correctnessbutalsodata,configuration,andmodelperformance implications[1,8,17,29].”
Barradas-Fernández, Á., Sanchéz-García, Á. J., Alonso-Ramirez, O., et al. (2025) · Mexican International Conference on Computer Science · doi:10.1109/enc68268.2025.11311899
“Automation and autonomous decision-making have brought AI and Machine Learning (ML) into software development, yet many systems still show deficiencies [9].”
Vallentin, A. G., Bech, N. R., & Paja, E. (2026) · Lecture notes in business information processing · doi:10.1007/978-3-032-26836-5_16
Testing Online Machine Learning Programs: A Scenario-Based Approach and Tools
Demougeot, M., Trouilhet, S., Arcangeli, J.-P., et al. (2026) · Communications in computer and information science · doi:10.1007/978-3-032-37353-3_3
Zong, R., Zhao, Y., He, S., & Rahim, M. A. B. U. (2026) · Communications in computer and information science · doi:10.1007/978-3-032-39763-8_6
- Authors
- Khomh, F., Adams, B., Cheng, J., Fokaefs, M., & Antoniol, G.
- Affiliations
- Polytechnique Montréal (CA)
- Venue
- IEEE Software
- Published
- 2018-09
- Volume
- 35
- Issue
- 5
- Pages
- 81-84
- ISSN
- 0740-7459 (ISSN-L)
- 1937-4194
- OpenAlex
- W2894409651
- Cited by
- 95
- References
- 0
- Citations by year
- 2026: 3 · 2025: 8 · 2024: 11 · 2023: 16 · 2022: 13 · 2021: 21 · 2020: 13 · 2019: 10
- FWCI
- 17.74
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- Locations
- IEEE Software (publishedVersion)
- PolyPublie (École Polytechnique de Montréal) (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Engineering › Safety, Risk, Reliability and Quality › Safety Systems Engineering in Autonomy
- Keywords
- software engineering for machine learning, artificial intelligence systems, industrial applications of AI
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
