Cited by 220Software Engineering Techniques and PracticesSoftware Engineering ResearchAdvanced Software Engineering Methodologies

The DOI matches its Crossref record; 22 of the 22 references it cites were found in a registry or an index.

A Taxonomy of Software Engineering Challenges for Machine Learning Systems: An Empirical Investigation

Open access

Lwakatare, Lucy Ellen, Raj, Aiswarya, Bosch, Jan, Olsson, Helena Holmström, Crnkovic, Ivica·2019·Lecture Notes in Business Information Processing

DOI 10.1007/978-3-030-19034-7_14

Open this paper in CoChat — free account Open PDF

Abstract

Artificial intelligence enabled systems have been an inevitable part of everyday life. However, efficient software engineering principles and processes need to be considered and extended when developing AI- enabled systems. The objective of this study is to identify and classify software engineering challenges that are faced by different companies when developing software-intensive systems that incorporate machine learning components. Using case study approach, we explored the development of machine learning systems from six different companies across various domains and identified main software engineering challenges. The challenges are mapped into a proposed taxonomy that depicts the evolution of use of ML components in software-intensive system in industrial settings. Our study provides insights to software engineering community and research to guide discussions and future research into applied machine learning.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Integrity signals

Registry

Crossref · 10.1007/978-3-030-19034-7_14

Cites

22 of 22 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Guidelines for conducting and reporting case study research in software engineering
Runeson, P., & Höst, M. (2009) · Empirical Software Engineering · doi:10.1007/s10664-008-9102-8 · Cited by 3,862
Untitled work
(2015) · neural information processing systems · Cited by 912
Applied Machine Learning at Facebook: A Datacenter Infrastructure Perspective
Hazelwood, K., Bird, S., Brooks, D., Chintala, S., Diril, U., Dzhulgakov, D., Fawzy, M., Jia, B., Jia, Y., Kalro, A., et al. (2018) · 2018 IEEE International Symposium on High Performance Computer Architecture (HPCA) · doi:10.1109/hpca.2018.00059 · Cited by 590
Guest Editors' Introduction: On Applied Research in Machine Learning
Provost, F. J., & Kohavi, R. (1998) · Machine Learning · doi:10.1023/a:1007442505281 · Cited by 385
Software Engineering Challenges of Deep Learning
Arpteg, A., Brinne, B., Crnkovic-Friis, L., & Bosch, J. (2018) · Euromicro Conference on Software Engineering and Advanced Applications (SEAA) · doi:10.1109/seaa.2018.00018 · Cited by 238
Self-citation
Large-scale machine learning at twitter
Lin, J., & Kolcz, A. (2012) · ACM SIGMOD International Conference on Management of Data (SIGMOD) · doi:10.1145/2213836.2213958 · Cited by 231
Data Management Challenges in Production Machine Learning
Polyzotis, N., Roy, S., Whang, S. E., & Zinkevich, M. (2017) · ACM SIGMOD International Conference on Management of Data (SIGMOD) · doi:10.1145/3035918.3054782 · Cited by 214
Machine Learning and Software Engineering
Zhang, D., & Tsai, J. J. P. (2003) · Software Quality Journal · doi:10.1023/a:1023760326768 · Cited by 184
Trials and tribulations of developers of intelligent systems: A field study
Hill, C., Bellamy, R., Erickson, T. D., & Burnett, M. (2016) · IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) · doi:10.1109/vlhcc.2016.7739680 · Cited by 115
Cascade Ranking for Operational E-commerce Search
Liu, S., Xiao, F., Ou, W., & Si, L. (2017) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3097983.3098011 · Cited by 111

Show all 22 references

Research works printed but not linked: 1

Lefortier, D., Truchet, A., de Rijke, M.: Sources of variability in large-scale machine learning systems. In: Machine Learning Systems (NIPS 2015 Workshop) (2015)

Sorted by citations

Items 101–200 of 220

Towards Artefact-based Requirements Engineering for Data-Centric SystemsPreprint
Chuprina, T., Méndez, D., & Wnuk, K. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.05233 · Cited by 10
Architecture Decisions in AI-based Systems Development: An Empirical Study
Zhang, B., Liu, T., Liang, P., Wang, C., Shahin, M., & Yu, J. (2023) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner56733.2023.00063 · Cited by 9
Themes of Building LLM-Based Applications for Production: A Practitioner's View
Mailach, A., Simon, S., Dorn, J., & Siegmund, N. (2025) · IEEE/ACM International Conference on AI Engineering – Software Engineering for AI (CAIN) · doi:10.1109/cain66642.2025.00011 · Cited by 9
Constructing reusable knowledge for machine learning projects based on project practices
Takeuchi, H., Imazaki, K., Kuno, N., Doi, T., & Motohashi, Y. (2022) · Intelligent Decision Technologies · doi:10.3233/idt-220252 · Cited by 9
Practitioners’ insights on machine-learning software engineering design patterns: a preliminary study
Washizaki, H., Takeuchi, H., Khomh, F., Natori, N., Doi, T., & Okuda, S. (2020) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsme46990.2020.00095 · Cited by 9
Reusability in Artificial Neural Networks
Ghofrani, J., Kozegar, E., Bozorgmehr, A., & Soorati, M. D. (2019) · International Software Product Line Conference (SPLC) · doi:10.1145/3307630.3342419 · Cited by 9
Questions for data scientists in software engineering: a replication
Huijgens, H., Rastogi, A., Mulders, E., Gousios, G., & Arie Van Deursen, A. (2020) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3368089.3409717 · Cited by 9
Technical risk model of machine learning based software project development - A multinational empirical study using modified Delphi-AHP method
Lin, C.-T., & Huang, S.-J. (2024) · Information and Software Technology · doi:10.1016/j.infsof.2024.107449 · Cited by 10
A Scoping Review and Assessment Framework for Technical Debt in the Development and Operation of AI/ML Competition Platforms
Sklavenitis, D., & Kalles, D. (2025) · Applied Sciences · doi:10.3390/app15137165 · Cited by 10
A Case Study on AI Engineering Practices: Developing an Autonomous Stock Trading System
Grote, M., & Bogner, J. (2023) · IEEE/ACM International Conference on AI Engineering – Software Engineering for AI (CAIN) · doi:10.1109/cain58948.2023.00032 · Cited by 8
Identifying Architectural Design Decisions for Achieving Green ML Serving
Durán, F., Martínez‐Fernández, S., Martínez, M., & Lago, P. (2024) · IEEE/ACM International Conference on AI Engineering – Software Engineering for AI (CAIN) · doi:10.1145/3644815.3644962 · Cited by 8
Unmasking Data Secrets: An Empirical Investigation into Data Smells and Their Impact on Data Quality
Recupito, G., Rapacciuolo, R., Nucci, D. D., & Palomba, F. (2024) · IEEE/ACM International Conference on AI Engineering – Software Engineering for AI (CAIN) · doi:10.1145/3644815.3644960 · Cited by 8
Human-centric Requirements Engineering for Artificial Intelligence Software Systems
Ahmad, K. (2021) · IEEE International Requirements Engineering Conference (RE) · doi:10.1109/re51729.2021.00070 · Cited by 8
Enterprise Architecture based Representation of Architecture and Design Patterns for Machine Learning Systems
Takeuchi, H., Doi, T., Washizaki, H., Okuda, S., & Yoshioka, N. (2021) · IEEE International Enterprise Distributed Object Computing Conference Workshops (EDOCW) · doi:10.1109/edocw52865.2021.00055 · Cited by 8
Challenges in migrating imperative deep learning programs to graph execution
Vélez, T. C., Khatchadourian, R., Bagherzadeh, M., & Raja, A. (2022) · International Conference on Mining Software Repositories (MSR) · doi:10.1145/3524842.3528455 · Cited by 8
Capturing Dependencies Within Machine Learning via a Formal Process Model
Ritz, F., Phan, T., Sedlmeier, A., Altmann, P. J., Wieghardt, J., Schmid, R., Sauer, H., Klein, C., Linnhoff–Popien, C., & Gabor, T. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19759-8_16 · Cited by 8
Developing Time Series Forecasting Models with Generative Large Language Models
Morales-García, J., Llanes, A., Arcas-Túnez, F., & Terroso-Sáenz, F. (2024) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3663485 · Cited by 7
Constructing Dependable Data-Driven Software With Machine Learning
Pahl, C., & Azimi, S. (2021) · IEEE Software · doi:10.1109/ms.2021.3067940 · Cited by 7
A Quality-driven Machine Learning Governance Architecture for Self-adaptive Edge Clouds
Pahl, C., Azimi, S., Barzegar, H. R., & Ioini, N. E. (2022) · International Conference on Cloud Computing and Services Science · doi:10.5220/0011107000003200 · Cited by 7
Exploring Ethical Requirements Elicitation for Applications in the Context of AI
de Cerqueira, J. S., & Canedo, E. (2022) · Anais Estendidos do XVIII Simpósio Brasileiro de Sistemas de Informação (SBSI Estendido 2022) · doi:10.5753/sbsi_estendido.2022.222269 · Cited by 8
Technical Debt Management in Industrial ML - State of Practice and Management Model Proposal
Wang, X., Schuster, H., Borrison, R., & Klöpper, B. (2023) · IEEE International Conference on Industrial Informatics (INDIN) · doi:10.1109/indin51400.2023.10217843 · Cited by 7
From Efficiency to Effectiveness: Delivering Business Value Through Software
Bosch, J. (2019) · Lecture notes in business information processing · doi:10.1007/978-3-030-33742-1_1 · Cited by 6
Self-citation
Reference Model for Agile Development of Machine Learning-based Service Systems
Takeuchi, H., Kaiya, H., Nakagawa, H., & Ogata, S. (2021) · doi:10.1109/apsecw53869.2021.00014 · Cited by 6
Practices and Infrastructures for Machine Learning Systems: An Interview Study in Finnish Organizations
Muiruri, D., Lwakatare, L. E., Nurminen, J. K., & Mikkonen, T. (2022) · Computer · doi:10.1109/mc.2022.3161161 · Cited by 6
Self-citation
Use of EA Models in Organizational AI Solution Development
Sandkuhl, K., & Rittelmeyer, J. D. (2022) · Lecture notes in business information processing · doi:10.1007/978-3-031-11520-2_10 · Cited by 6
Chapter 13 Engineering AI Systems
Bosch, J., Olsson, H. H., & Crnković, I. (2021) · Springer eBooks · doi:10.1007/978-3-031-10873-0_18 · Cited by 8
Self-citation
Practice-based Collection of Bad Smells in Machine Learning Projects
Takeuchi, H., Kaiya, H., Nakagawa, H., & Ogata, S. (2023) · Procedia Computer Science · doi:10.1016/j.procs.2023.10.036 · Cited by 6
Knowledge-based Adequacy assessment Approach to support AI adoption
Jahić, J., Roitsch, R., & Grzymkowski, Ł. (2021) · IEEE International Conference on Software Architecture Companion (ICSA-C) · doi:10.1109/icsa-c52384.2021.00008 · Cited by 7
On the Impact of ML use cases on Industrial Data Pipelines
Raj, M. A., Bosch, J., Olsson, H. H., & Jansson, A. (2021) · Asia-Pacific Software Engineering Conference (APSEC) · doi:10.1109/apsec53868.2021.00053 · Cited by 6
Self-citation
When Domains Collide: An Activity Theory Exploration of Cross-Disciplinary Collaboration
Feng, Z., Zimmermann, T., Pisani, L., Gooley, C. J., Wander, J. D., & Sarma, A. (2025) · Proceedings of the ... ACM-IEEE International Symposium on Empirical Software Engineering and Measurement/International Symposium on Empirical Software Engineering and Measurement · doi:10.1109/esem64174.2025.00017 · Cited by 5
The Five Purposes of Value Modeling
Olsson, H. H., & Bosch, J. (2020) · Euromicro Conference on Software Engineering and Advanced Applications (SEAA) · doi:10.1109/seaa51224.2020.00027 · Cited by 5
Self-citation
Holistic QA: Software Quality Assurance for the Machine Learning Era
Downing, S., & Badar, M. A. (2023) · International Conference on Industrial Engineering and Operations Management (IEOM) · doi:10.46254/na07.20220050 · Cited by 5
ITSM BASED CHANGE MANAGEMENT AUTOMATION IN CLOUD ENVIRONMENTS: A CROSS SECTOR EMPIRICAL STUDY
Jyoti, S. N. (2025) · doi:10.63125/xvjst226 · Cited by 5
Developing a maturity-based workflow for the implementation of ML-applications using the example of a demand forecast
Schreckenberg, F., & Moroff, N. U. (2021) · Procedia Manufacturing · doi:10.1016/j.promfg.2021.07.006 · Cited by 5
Discovering Artificial Intelligence Implementation and Insights for Lean Production
Kassem, B., Costa, F., & Staudacher, A. P. (2021) · IFIP advances in information and communication technology · doi:10.1007/978-3-030-92934-3_18 · Cited by 5
Analytical Review of Intelligent Intrusion Detection Systems Based on Federated Learning: Advantages and Open Challenges
Novikova, E. S., Владимировна, Ф. Е., Kotenko, I. V., & Kholod, I. I. (2023) · Informatics and Automation · doi:10.15622/ia.22.5.4 · Cited by 5
Towards a Digital Business Operating System
Bosch, J. (2019) · International Conference on Research Challenges in Information Science (RCIS) · doi:10.1109/rcis.2019.8877053 · Cited by 5
Self-citation
Towards Productizing AI/ML Models: An Industry Perspective from Data Scientists
Lanubile, F., Calefato, F., Quaranta, L., Amoruso, M., Fumarola, F., & Filannino, M. (2021) · IEEE/ACM International Conference on AI Engineering – Software Engineering for AI (CAIN) · doi:10.1109/wain52551.2021.00027 · Cited by 5
PlanTogether: Facilitating AI Application Planning Using Information Graphs and Large Language Models
Kim, D. H., Jeong, D., Yadgarova, S., Shin, H., Son, J., Subramonyam, H., & Kim, J. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714044 · Cited by 4
Component-based Approach to Software Engineering of Machine Learning-enabled Systems
Indykov, V. (2024) · IEEE/ACM International Conference on AI Engineering – Software Engineering for AI (CAIN) · doi:10.1145/3644815.3644976 · Cited by 4
Software Engineering Taxonomy Reviews
Mubarakah, N., & Soeharwinto (2020) · International Conference on Electrical, Telecommunication and Computer Engineering (ELTICOM) · doi:10.1109/elticom50775.2020.9230473 · Cited by 4
Engineering an Intelligent Essay Scoring and Feedback System: An Experience Report
Chadda, A., Song, K., Chandrasekar, R., & Gorton, I. (2021) · IEEE/ACM International Conference on AI Engineering – Software Engineering for AI (CAIN) · doi:10.1109/wain52551.2021.00029 · Cited by 4
The Secrets of Data Science Deployments
Fayyad, U. M. (2022) · IEEE Intelligent Systems · doi:10.1109/mis.2022.3196612 · Cited by 4
Enabling Cognitive Manufacturing in Heterogeneous Industrial Automation Systems
Van, H. M., Dreiser, M., da Silva, A. S., & Weiß, G. (2022) · IEEE International Conference on Cognitive Informatics & Cognitive Computing (ICCI*CC) · doi:10.1109/iccicc57084.2022.10101573 · Cited by 4
Dynamic Data Management for Machine Learning in Embedded Systems: A Case Study
Ouhaichi, H., Olsson, H. H., & Bosch, J. (2019) · Lecture notes in business information processing · doi:10.1007/978-3-030-33742-1_12 · Cited by 4
Self-citation
The Assessment of Importance of Selected Issues of Software Engineering, IT Project Management, and Programming Paradigms Based on Graphical AHP and Fuzzy C-Means
Karczmarek, P., Pedrycz, W., Czerwiński, D., & Kiersztyn, A. (2020) · IEEE International Conference on Fuzzy Systems (FUZZ-IEEE) · doi:10.1109/fuzz48607.2020.9177591 · Cited by 4
Handling Non-Fuctional Requirements in IoT-based Machine Learning Systems
Khan, A., Siddiqui, I. F., Shaikh, M., Anwar, S., & Shaikh, M. (2022) · Joint International Conference on Digital Arts, Media and Technology with ECTI Northern Section Conference on Electrical, Electronics, Computer and Telecommunications Engineering (ECTI DAMT & NCON) · doi:10.1109/ectidamtncon53731.2022.9720403 · Cited by 4
A Drift Handling Approach for Self-Adaptive ML Software in Scalable Industrial Processes
Bayram, F., Ahmed, B. S., Hallin, E., & Engman, A. (2022) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3551349.3559495 · Cited by 4
Design Methods and Processes for ML/DL models
John, M. M. (2021) · doi:10.24834/isbn.9789178771998 · Cited by 4
Towards designing a flexible multimodal learning analytics system
Ouhaichi, H. (2022) · doi:10.24834/isbn.9789178772988 · Cited by 4
Proceedings of the 2024 IEEE/ACM 17th International Conference on Cooperative and Human Aspects of Software Engineering
(2024) · doi:10.1145/3641822 · Cited by 4
Enterprise Architecture-Based Project Model for AI Service System Development
Takeuchi, H. (2022) · Artificial intelligence-enhanced software and systems engineering · doi:10.1007/978-3-031-08202-3_7 · Cited by 3
Assessing the Quality of Computational Notebooks for a Frictionless Transition from Exploration to Production
Quaranta, L. (2022) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3510454.3517055 · Cited by 3
Chapter 8 Recognizing Lines of Code Violating Company-Specific Coding Guidelines Using Machine Learning
Ochodek, M., Hebig, R., Meding, W., Frost, G., & Staron, M. (2019) · Springer eBooks · doi:10.1007/978-3-031-10873-0_11 · Cited by 3
Investigating Bugs in AI-Infused Systems: Analysis and Proposed Taxonomy
Kassab, M., DeFranco, J. F., & Laplante, P. A. (2022) · IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW) · doi:10.1109/issrew55968.2022.00094 · Cited by 3
Characterizing Technical Debt and Antipatterns in AI-Based Systems: A Systematic Mapping Study
Bogner, J., Verdecchia, R., & Gerostathopoulos, I. (2021) · Figshare · doi:10.5281/zenodo.4457215 · Cited by 3
Hey ML, what can you do for me?
Pastorino, J., & Biswas, A. K. (2020) · IEEE International Conference on Artificial Intelligence and Knowledge Engineering (AIKE) · doi:10.1109/aike48582.2020.00023 · Cited by 3
Re-orienting Toward the Science of the Artificial: Engineering AI Systems
Russell, S., Jalaian, B. A., & Moskowitz, I. S. (2021) · Springer eBooks · doi:10.1007/978-3-030-77283-3_8 · Cited by 3
A Software Engineering Perspective on Building Production-Ready Machine Learning Systems
Heck, P., Schouten, G., & Cruz, L. (2021) · Advances in business information systems and analytics book series · doi:10.4018/978-1-7998-6985-6.ch002 · Cited by 3
Educating AI Software Engineers: Challenges and Opportunities
Bublin, M., Schefer-Wenzl, S., & Miladinović, I. (2022) · Lecture notes in networks and systems · doi:10.1007/978-3-030-93907-6_26 · Cited by 3
More Engineering, No Silos: Rethinking Processes and Interfaces in Collaboration between Interdisciplinary Teams for Machine Learning Projects.Preprint
Nahar, N., Zhou, S., Lewis, G. A., & Kästner, C. (2021) · arXiv (Cornell University) · Cited by 3
IDENTIFYING CONCERNS WHEN SPECIFYING MACHINE LEARNING-ENABLED SYSTEMS: A PERSPECTIVE-BASED APPROACH
Villamizar, H. (2023) · doi:10.17771/pucrio.acad.65972 · Cited by 3
Chapter 10 Requirements Engineering Challenges and Practices in Large-Scale Agile System Development
Kasauli, R., Knauss, E., Horkoff, J., Liebel, G., & Neto, F. G. D. O. (2020) · Springer eBooks · doi:10.1007/978-3-031-10873-0_14 · Cited by 2
Chapter 4 Technical Debt Tracking: Current State of Practice: A Survey and Multiple Case Study in 15 Large Organizations
Martini, A., Besker, T., & Bosch, J. (2018) · Springer eBooks · doi:10.1007/978-3-031-10873-0_6 · Cited by 3
Self-citation
Liquifying Quantum-Classical Software-Intensive System of Systems
Aparicio-Morales, Á. M., Haghparast, M., Mäkitalo, N., García-Alonso, J. M., Berrocal, J., Stirbu, V., Mikkonen, T., & Murillo, J. M. (2024) · IEEE International Conference on Software Analysis, Evolution and Reengineering - Companion (SANER-C) · doi:10.1109/saner-c62648.2024.00028 · Cited by 2
PRISE: A Framework for AI Product Incubation From Concept to Implementation
Wang, B., Li, Y., & Lei, W. (2025) · IEEE Access · doi:10.1109/access.2025.3587943 · Cited by 2
DRGM: Data Requirement Goal Modeling for ML-Based Systems
Yamani, A. Z., AlAmoudi, N., Albelali, S., Baslyman, M., & Hassine, J. (2025) · IEEE International Requirements Engineering Conference Workshops (REW) · doi:10.1109/rew66121.2025.00013 · Cited by 3
Human-in-the-loop governance of artificial intelligence in cardiology: From ethical principles to operational paradigms
Sharma, K., Sharma, P., & Sharma, P. (2026) · Indian Heart Journal · doi:10.1016/j.ihj.2026.06.002 · Cited by 2
A Formal Approach to Engineering AI-based Systems: An Introduction
Kannan, H. (2023) · AIAA SciTech Forum · doi:10.2514/6.2023-1115 · Cited by 2
Software Architectures for AI Systems: State of Practice and Challenges
Gorton, I., Khomh, F., Lenarduzzi, V., Menghi, C., & Roman, D. (2023) · Springer eBooks · doi:10.1007/978-3-031-36847-9_2 · Cited by 2
Confiabilidade e Validade da Avaliação do Desempenho de Aprendizagem de Machine Learning na Educação Básica
Rauber, M. F., Garcia, A. B., von Wangenheim, C. G., Borgatto, A. F., Martins, R. M., & Hauck, J. C. R. (2022) · Anais do ... Simpósio Brasileiro de Informática na Educação/Anais do Simpósio Brasileiro de Informática na Educação · doi:10.5753/sbie.2022.224688 · Cited by 2
Uma Análise Detalhada do Desempenho de Aprendizagem ensinando Machine Learning na Educação Básica aplicando a Teoria de Resposta ao Item
Rauber, M. F., von Wangenheim, C. G., Borgatto, A. F., & Martins, R. M. (2023) · Revista Brasileira de Informática na Educação · doi:10.5753/rbie.2023.3442 · Cited by 1
Use ChatGPT to Inspect Software Developed by Novice Programmers
Ibarra-Torres, F., & Barona-Pico, V. (2024) · Lecture notes in networks and systems · doi:10.1007/978-3-031-70981-4_8 · Cited by 2
Project Practice-Based Anti-patterns for Machine Learning Projects
Takeuchi, H., Kaiya, H., Nakagawa, H., & Ogata, S. (2025) · Learning and analytics in intelligent systems · doi:10.1007/978-3-031-98410-5_16 · Cited by 2
Chapter 3 Efficient and Effective Exploratory Testing of Large-Scale Software Systems
Mårtensson, T., Ståhl, D., Martini, A., & Bosch, J. (2021) · Springer eBooks · doi:10.1007/978-3-031-10873-0_4 · Cited by 2
Self-citation
Análise do desempenho de aprendizagem de Machine Learning na Educação Básica aplicando a Teoria de Resposta ao Item
Rauber, M. F., von Wangenheim, C. G., Borgatto, A. F., & Martins, R. M. (2023) · doi:10.5753/educomp.2023.228159 · Cited by 2
Understanding Artificial Intelligence Through Its Applications and Concerns
Maki, H. A., Mubarak, M. A., & Bakır, A. (2023) · Internet of things · doi:10.1007/978-3-031-35525-7_9 · Cited by 2
Development and Deployment Challenges of Machine Learning Systems
Honkanen, T., O’Dwyer, J., Salminen, V., & Jussila, J. (2024) · Communications in computer and information science · doi:10.1007/978-3-031-64359-0_15 · Cited by 2
Practices and Infrastructures for ML Systems -- An Interview StudyPreprint
Muiruri, D., Lwakatare, L. E., Nurminen, J. K., & Mikkonen, T. (2021) · doi:10.36227/techrxiv.16939192.v1 · Cited by 2
Self-citation
Designing Data Science Software for Social Care Organisations
Reinmund, T. (2023) · CHI Extended Abstracts · doi:10.1145/3544549.3577062 · Cited by 1
Perspectives On Machine Learning Inference Serving in Real‐World Settings
Muiruri, D., Lwakatare, L. E., Nurminen, J. K., & Mikkonen, T. (2026) · Software Practice and Experience · doi:10.1002/spe.70069 · Cited by 1
Self-citation
Chapter 7 MESRAM – A Method for Assessing Robustness of Measurement Programs in Large Software Development Organizations and Its Industrial Evaluation
Staron, M., & Meding, W. (2015) · Springer eBooks · doi:10.1007/978-3-031-10873-0_10 · Cited by 1
Collecting Insights and Developing Patterns for Machine Learning Projects Based on Project Practices
Takeuchi, H., Imazaki, K., Kuno, N., Doi, T., & Motohashi, Y. (2023) · Learning and analytics in intelligent systems · doi:10.1007/978-3-031-17583-1_5 · Cited by 1
Architecting Machine Learning Systems: Which Parts are the Architect's Pain?
Jepsen, S. C., Nielsen, A. O., & Kjærgaard, M. B. (2024) · IEEE International Conference on Software Architecture Companion (ICSA-C) · doi:10.1109/icsa-c63560.2024.00057 · Cited by 1
Introduction to the Metrics Theme
Staron, M. (2022) · Springer eBooks · doi:10.1007/978-3-031-10873-0_9 · Cited by 1
Chapter 2 Modeling Continuous Integration Practice Differences in Industry Software Development
Ståhl, D., & Bosch, J. (2013) · Springer eBooks · doi:10.1007/978-3-031-10873-0_3 · Cited by 1
Self-citation
Introduction to the AI Engineering Theme
Bosch, J. (2022) · Springer eBooks · doi:10.1007/978-3-031-10873-0_17 · Cited by 1
Self-citation
Analyzing Organizational Processes in Machine Learning Projects: Exploring Modeling Approaches
Sothilingam, R. (2020) · TSpace (University of Toronto) · Cited by 1
Practices and Infrastructures for ML Systems -- An Interview Study in Finnish OrganizationsPreprint
Muiruri, D., Lwakatare, L. E., Nurminen, J. K., & Mikkonen, T. (2022) · doi:10.36227/techrxiv.16939192.v2 · Cited by 1
Self-citation
Practices and Infrastructures for ML Systems – An Interview Study in Finnish OrganizationsPreprint
Muiruri, D., Lwakatare, L. E., Nurminen, J. K., & Mikkonen, T. (2021) · doi:10.36227/techrxiv.16939192
Self-citation
Practices and Infrastructures for ML Systems -- An Interview Study in Finnish OrganizationsPreprint
Muiruri, D., Lwakatare, L. E., Nurminen, J. K., & Mikkonen, T. (2022) · doi:10.36227/techrxiv.16939192.v3
Self-citation
Method for Assessing Social Acceptability of AI Service Systems
Takeuchi, H., Ichitsuka, A., Iino, T., Ishikawa, S., Maeda, M., & Miyazawa, Y. (2022) · Smart innovation, systems and technologies · doi:10.1007/978-981-19-3455-1_17
TBD filler
(2022) · Computer · doi:10.1109/mc.2022.3195369
Introduction to the Continuous Architecture Theme
Berger, C., & Carlson, J. (2022) · Springer eBooks · doi:10.1007/978-3-031-10873-0_5
Introduction to the Continuous Delivery Theme
Sandahl, K. (2022) · Springer eBooks · doi:10.1007/978-3-031-10873-0_1
Introduction to the Customer Data and Ecosystem-Driven Development Theme
Olsson, H. H. (2022) · Springer eBooks · doi:10.1007/978-3-031-10873-0_13
Self-citation
Chapter 11 Experimentation for Business-to-Business Mission-Critical Systems: A Case Study
Mattos, D. I., Dakkak, A., Bosch, J., & Olsson, H. H. (2020) · Springer eBooks · doi:10.1007/978-3-031-10873-0_15
Self-citation
Chapter 12 The Evolution of Continuous Experimentation in Software Product Development: From Data to a Data-Driven Organization at Scale
Fabijan, A., Dmitriev, P., Bosch, J., & Olsson, H. H. (2017) · Springer eBooks · doi:10.1007/978-3-031-10873-0_16
Self-citation
Chapter 9 SimSAX: A Measure of Project Similarity Based on Symbolic Approximation Method and Software Defect Inflow
Ochodek, M., Staron, M., & Meding, W. (2019) · Springer eBooks · doi:10.1007/978-3-031-10873-0_12
Chapter 5 Expectations and Challenges from Scaling Agile in Mechatronics-Driven Companies – A Comparative Case Study
Berger, C., & Eklund, U. (2015) · Springer eBooks · doi:10.1007/978-3-031-10873-0_7

Authors: Lwakatare, L. E., Raj, A., Bosch, J., Olsson, H. H., & Crnkovic, I.
Affiliations: Chalmers University of Technology (SE)
Malmö University (SE)
Corresponding author: Lwakatare, Lucy Ellen
ORCID: Lwakatare, Lucy Ellen
Bosch, Jan
Olsson, Helena Holmström
Venue: Lecture Notes in Business Information Processing
Publisher: Springer International Publishing
Published: 2019
Pages: 227-243
ISSN: 1865-1348 (ISSN-L)
1865-1356
DOI: 10.1007/978-3-030-19034-7_14
OpenAlex: W2944264045
Cited by: 214
References: 22
Citations by year: 2026: 4 · 2025: 20 · 2024: 27 · 2023: 31 · 2022: 43 · 2021: 44 · 2020: 29 · 2019: 10
FWCI: 25.09
Type: conference paper
Language: en
Open access: hybrid
doi.org
Licence: cc-by
creativecommons.org/licenses/by/4.0 (tdm)
Locations: Lecture notes in business information processing (publishedVersion, cc-by)
KTH Publication Database DiVA (KTH Royal Institute of Technology) (submittedVersion, other-oa)
Malmö University Publications (Malmö University) (submittedVersion)
Chalmers Research (Chalmers University of Technology) (submittedVersion)
Topics: Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
Physical Sciences › Computer Science › Information Systems › Software Engineering Research
Physical Sciences › Computer Science › Software › Advanced Software Engineering Methodologies
Keywords: software engineering challenges, machine learning systems, artificial intelligence systems, software-intensive systems, industrial software development, applied machine learning

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

A Taxonomy of Software Engineering Challenges for Machine Learning Systems: An Empirical Investigation

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Self-check

Citations

Cites

Cited by

Related

Metadata

Record

About these checks