A software engineering perspective on engineering machine learning systems: State of the art and challenges
Open accessAI snapshot
Written from the paper’s full text.
TL;DR
This systematic literature review synthesizes 141 SE-venue primary studies (2007–Dec 2020) on engineering machine learning systems. It finds that ML systems’ non-deterministic nature complicates all SE aspects, no SE aspect yet has mature tools and techniques, and testing is the most studied area but still lacks strong experimental proof.
Key findings
- The review analyzed 141 primary studies from SE venues; research interest increased sharply from 2018 onward, and 91% of studies appeared in conferences/workshops (129 papers) versus 12 in journals.
- Testing is by far the most popular area among researchers, but even for testing ML systems engineers have only tool prototypes and solution proposals with weak experimental proof; many challenges were identified through surveys and interviews.
- Among 93 experiment/case-study papers, 65 (70%) used only DL algorithms, 19 used traditional ML, 4 used both, and 5 did not report algorithms; image classification was the most popular application scenario, used in 57 studies.
- No SE aspect has a mature set of tools and techniques, and the paper calls for experiments and case studies, ideally in industrial environments, to better understand challenges and propose solutions.
Methods
- Systematic literature review following established guidelines, with database search (ACM, IEEE Xplore, ScienceDirect, Springer, Wiley, Google Scholar), backward/forward snowballing, and manual search of ICSE, ESEC/FSE, AIRE, and RAISE.
- Inclusion/exclusion criteria restricted papers to SE venues and to those involving at least one SE challenge of engineering ML systems; after de-duplication and quality assessment, the final pool was 141 studies.
- Quantitative analysis reported frequencies/percentages for research methods and application scenarios/datasets; qualitative open coding analyzed challenges and solutions, classified under SWEBOK-based SE knowledge areas.
Limitations
- Paper-stated: scope is limited to studies published until 31 December 2020, peer-reviewed academic venues (grey literature excluded), and SE venues only (AI/ML/DL/Data Science/Data Management venues excluded); primary study selection, data extraction, and synthesis are subject to researcher bias, which may be higher in a single-author SLR.
- Paper-stated: single data extraction can produce more errors than double data extraction, and the author notes that selection only from SE venues may miss relevant studies from other venues.
- Evident from text: the search used broad terms and a “search saturation effect,” checking only the first 200 results per query in each database unless results 190–200 still looked relevant, which may limit recall despite snowballing and manual search.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
0 of 326 cited sources checked · 0 supported · 0 flagged
- Findingown data
This study identified more papers published in SE venues (141) than Kumeno (2019) (47).
“Unlike Kumeno (2019), I was able to identify more papers published in SE venues (141 vs. 47) and provided a taxonomy of challenges and proposed solutions classified under SE's main knowledge areas.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Kumeno (2019)
Not checked in the free run.
- Findingown data
Testing is by far the most popular area among researchers.
“Testing is by far the most popular area among researchers.”
Quote in the paperNot a substantial claim - Findingnone given
Many of the challenges of ML systems engineering were identified through surveys and interviews.
“Many of the challenges of ML systems engineering were identified through surveys and interviews.”
Quote in the paperNot a substantial claim - Findingown data
Despite increasing interest from 2018 onwards, none of the SE aspects have a mature set of tools and techniques.
“Despite increasing interest from 2018 onwards, the results reveal that none of the SE aspects have a mature set of tools and techniques.”
Quote in the paperSubstantial claim - Findingown data
This review is the most contemporary and up-to-date review.
“This review on the state-of-the-art is the most contemporary as it is the most up-to-date review.”
Quote in the paperSubstantial claim
- Findingown data
Even for testing ML systems, engineers have only some tool prototypes and solution proposals with weak experimental proof.
“Even for testing ML systems, engineers have only some tool prototypes and solution proposals with weak experimental proof.”
Quote in the paperSubstantial claim - Findingown data
The number of papers analyzed in this study (141) is higher than in existing reviews addressing SE aspects.
“The included number of papers in Kumeno (2019), Lwakatare et al. (2020), Nascimento et al. (2020), and this study are 115, 72, 55, and 141, respectively.”
Quote in the paperSubstantial claim0 of 3 sources checked- cites Kumeno (2019)
Not checked in the free run.
- cites Lwakatare et al. (2020) source
Not checked in the free run.
- cites Nascimento et al. (2020)
Not checked in the free run.
- Findingown data
Eight studies in the primary study pool conducted an experiment for predicting steering wheel angle.
“Eight studies in our primary study pool (Haq et al., 2020;Harel-Canada et al., 2020;Jahangirova and Tonella, 2020;Li et al., 2019b;Stocco and Tonella, 2020;Stocco et al., 2020;Tian et al., 2018;Zhang et al., 2018a) conducted an experiment for predicting steering wheel angle.”
Quote in the paperNot a substantial claim0 of 3 sources checked- cites Haq et al., 2020
Not checked in the free run.
- cites Harel-Canada et al., 2020
Not checked in the free run.
- cites Jahangirova and Tonella, 2020
Not checked in the free run.
- Findingreasoning
Image classification is the most popular application scenario used in 57 primary studies.
“Image classification is the most popular application scenario used in 57 primary studies.”
Quote in the paperNot a substantial claim - Findingnone given
Twenty studies included a classification scenario other than image classification.
“Twenty studies included a classification scenario other than image classification.”
Quote in the paperNot a substantial claim - Findingown data
The third most popular application scenario (used in 13 studies) is autonomous driving, following image and other types of classification scenarios.
“The third most popular application scenario (used in 13 studies) is autonomous driving, following image and other types of classification scenarios.”
Quote in the paperNot a substantial claim - Findingreasoning
DL algorithms were used in all primary studies involving autonomous driving, continuous value prediction, object detection, and mountain car problem scenarios.
“In line with these findings, DL algorithms were used in all primary studies involving autonomous driving, continuous value prediction, object detection, and mountain car problem scenarios.”
Quote in the paperNot a substantial claim - Findingown data
One hundred twenty one out of 141 primary studies address only one knowledge area.
“One hundred twenty one out of 141 primary studies address only one knowledge area.”
Quote in the paperNot a substantial claim - Findingown data
Most of the empirical studies employed experiments.
“Most of the empirical studies employed experiments.”
Quote in the paperNot a substantial claim - Findingown data
Five studies addressed testing and maintenance aspects of ML systems by analyzing ML frameworks or libraries involved in these systems.
“Five studies addressed testing and maintenance aspects of ML systems by analyzing ML frameworks or libraries involved in these systems.”
Quote in the paperNot a substantial claim - Findingreasoning
Nearly all of the image classification scenarios were also experimented using DL algorithms.
“In addition, nearly all of the image classification scenarios were also experimented using DL algorithms.”
Quote in the paperNot a substantial claim - Findingown data
Four studies involved an application scenario to predict a numerical value for an observation.
“Four studies involved an application scenario to predict a numerical value for an observation.”
Quote in the paperNot a substantial claim - Findingreasoning
For the classification scenarios excluding image classification, the researchers mostly used traditional ML algorithms.
“For the classification scenarios excluding image classification, the researchers mostly used traditional ML algorithms.”
Quote in the paperNot a substantial claim - Findingown data
The studies that used an experiment or a case study as a research method involved one or more application scenarios to observe SE challenges and validate a solution.
“The studies that used an experiment or a case study as a research method involved one or more application scenarios to observe SE challenges and validate a solution.”
Quote in the paperNot a substantial claim - Findingnone given
Based on the final paper pool, most challenges in developing ML systems manifest in issues related to data; models; dependencies; infrastructure and tools; ML algorithms, techniques, and libraries; and reuse.
“Based on the final paper pool, most challenges in developing ML systems manifest in issues related to data; models; dependencies; infrastructure and tools; ML algorithms, techniques, and libraries; and reuse.”
Quote in the paperSubstantial claim - Findingown data
Twenty-four studies include custom datasets constructed by researchers (including synthetic datasets) or privately owned by a company.
“Twenty-four studies include custom datasets constructed by researchers (including synthetic datasets) or privately owned by a company.”
Quote in the paperNot a substantial claim - Findingcitation
Recently, researchers started to explore conflicts between specific quality attributes, such as the trade-off between fairness and performance or accuracy.
“Recently, researchers started to explore conflicts between specific quality attributes, such as the trade-off between fairness and performance or accuracy (Kamishima et al., 2011).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Kamishima et al., 2011
Not checked in the free run.
- Findingown data
Eleven studies used Udacity self-driving car challenge dataset for experiments in autonomous driving domain for various tasks such as predicting steering wheel angle.
“Eleven studies used Udacity self-driving car challenge dataset for experiments in autonomous driving domain for various tasks such as predicting steering wheel angle (Kim et al., 2019;Li et al., 2019b;Tian et al., 2018).”
Quote in the paperNot a substantial claim0 of 3 sources checked- cites Kim et al., 2019
Not checked in the free run.
- cites Li et al., 2019b
Not checked in the free run.
- cites Tian et al., 2018
Not checked in the free run.
- Findingown data
Twenty-nine datasets were only used once in an experiment or case study.
“Twentynine datasets were only used once in an experiment or case study.”
Quote in the paperNot a substantial claim - Findingreasoning
Most of the datasets in Table 7 are the datasets for image classification, such as MNIST, CIFAR-10, ImageNet, Fashion-MNIST, SVHN, and IMDB-WIKI dataset.
“Since most of the studies are focused on image classification problems, most of the datasets in Table 7 are the datasets for image classification, such as MNIST (LeCun et al., 1998), CIFAR-10 ( Krizhevsky and Hinton, 2009), ImageNet (Fei-Fei, 2010), Fashion-MNIST (Xiao et al., 2017), SVHN (Netzer et al., 2011), and IMDB-WIKI dataset.”
Quote in the paperNot a substantial claim0 of 3 sources checked- cites LeCun et al., 1998 source
Not checked in the free run.
- cites Krizhevsky and Hinton, 2009
Not checked in the free run.
- cites Fei-Fei, 2010
Not checked in the free run.
- Findingnone given
Although transfer learning is an attractive topic in ML, it has not been addressed by the studies in the final paper pool.
“Although transfer learning is an attractive topic in ML, it has not been addressed by the studies in the final paper pool.”
Quote in the paperNot a substantial claim - Findingcitation
Developers usually reported that analyzing and understanding the structure and behavior of ML models (mainly neural networks) is very difficult.
“Developers usually reported that analyzing and understanding the structure and behavior of ML models (mainly neural networks) is very difficult (Amershi et al., 2019;Arpteg et al., 2018;Hill et al., 2016;Xie et al., 2019a;Zhang et al., 2019b).”
Quote in the paperSubstantial claim0 of 3 sources checked- cites Amershi et al., 2019
Not checked in the free run.
- cites Arpteg et al., 2018
Not checked in the free run.
- cites Hill et al., 2016
Not checked in the free run.
- Findingown data
The papers in the review pool mainly identify three interrelated groups of challenges in maintenance and configuration management.
“The papers in my pool mainly identify three groups of challenges, which are interrelated:”
Quote in the paperNot a substantial claim - Findingown data
Testing & Quality is the most focused knowledge area by academia and industry.
“Testing & Quality is the most focused knowledge area by academia and industry.”
Quote in the paperNot a substantial claim - Findingown data
Seven papers in the final pool address the challenges in two main categories from an organizational perspective: having the required skill sets and having a team working harmoniously.
“Seven papers in the final pool address the challenges in two main categories from an organizational perspective: Having the required skill sets and having a team working harmoniously.”
Quote in the paperNot a substantial claim - Findingown data
There was no paper or one paper per year until 2017.
“There was no paper or one paper per year until 2017.”
Quote in the paperNot a substantial claim - Findingown data
Experiments are mostly used for testing and quality aspects.
“As pointed out under RQ1, experiments are mostly used for testing and quality aspects.”
Quote in the paperSubstantial claim - Findingown data
Researchers have mainly used classification problems for their experiments and case studies.
“The answer to RQ2 indicates that researchers have mainly used classification problems for their experiments and case studies.”
Quote in the paperSubstantial claim - Findingown data
Many challenges have not been addressed.
“As Table 8 reveals, many challenges have not been addressed.”
Quote in the paperSubstantial claim - Findingnone given
There are a few studies targeting development and maintenance.
“There are a few studies targeting development and maintenance.”
Quote in the paperNot a substantial claim - Findingown data
Our primary study pool does not include any practices for managing ML systems development projects.
“While there are some examples of using ML algorithms for project management (two examples in ML for SE by Nassif et al. (2016) and Pospieszny et al. (2018)), our primary study pool does not include any practices for managing ML systems development projects.”
Quote in the paperSubstantial claim - Findingnone given
Even for the difficulties addressed, the proposed solutions are mostly conceptual or implemented as a prototype tool without industrial use.
“Moreover, even for the difficulties addressed, the proposed solutions are mostly conceptual or implemented as a prototype tool without industrial use.”
Quote in the paperSubstantial claim - Findingown data
In the reviewed literature, research on engineering ML systems is mostly dominated by academicians.
“As we can see in Figure 10, this research topic is mostly dominated by academicians, although engineering ML systems is a highly practical area.”
Quote in the paperSubstantial claim - Findingown data
The paper pool does not include any study that uses experiment to address requirements-and design-related challenges.
“The paper pool does not include any study that uses experiment to address requirements-and design-related challenges.”
Quote in the paperSubstantial claim - Findingown data
Researchers also conducted experiments on development & tools and maintenance & configuration management knowledge areas.
“Researchers also conducted experiments on development & tools and maintenance & configuration management knowledge areas.”
Quote in the paperNot a substantial claim - Findingown data
The dominant research method, i.e., experiment, is primarily used in studies addressing testing and quality.
“The dominant research method, i.e., experiment, is primarily used in studies addressing testing and quality.”
Quote in the paperNot a substantial claim - Findingown data
As a result of thematic and statistical analysis, the researchers have identified some difficulties in development & tools, test & quality, and organizational topics.
“As a result of thematic and statistical analysis, the researchers have identified some difficulties in development & tools, test & quality, and organizational topics.”
Quote in the paperSubstantial claim - Findingown data
The final primary study pool included 141 individual results.
“After excluding one study (found via DB search), the final primary study pool included 141 individual results.”
Quote in the paperNot a substantial claim - Findingown data
Five papers did not report the algorithms used.
“Five papers did not report the algorithms used.”
Quote in the paperNot a substantial claim - Findingown data
Among 93 papers, 65 papers (70%) used only DL algorithms.
“Among 93 papers, 65 papers (70%) used only DL algorithms.”
Quote in the paperNot a substantial claim - Interpretationreasoning
ML systems engineering in real-world settings is challenging because it adds additional complexity to engineering traditional software.
“ML systems engineering in real-world settings is challenging since it adds additional complexity to engineering”
Quote in the paperSubstantial claim - Premisecitation
47% of AI projects remain prototypes due to the lack of tools to develop and maintain a production-grade AI system.
“Despite several promising examples, 47% of AI projects remain prototypes due to the lack of the tools to develop and maintain a production-grade AI system, according to Gartner Research (Gartner, 2020).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Gartner, 2020
Not checked in the free run.
- Premisenone given
To the best of the author's knowledge, Kumeno (2019) published the first SLR addressing the challenges of engineering ML systems.
“To the best of my knowledge, Kumeno (2019) published the first SLR addressing the challenges of engineering ML systems.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Kumeno (2019)
Not checked in the free run.
- Interpretationnone given
The non-deterministic nature of ML systems complicates all SE aspects of engineering ML systems.
“The non-deterministic nature of ML systems complicates all SE aspects of engineering ML systems.”
Quote in the paperNot a substantial claim - Premisecitation
For scenarios where there is a big gap between the inputs and the outputs, DL models are able to perform much better than traditional ML models.
“Hence, for scenarios where there is a big gap between the inputs (such as images) and the outputs (such as objects), DL models are able to perform much better than traditional ML models (Zhou et al., 2018).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Zhou et al., 2018 source
Not checked in the free run.
- Premisecitation
Large labelled datasets are required to train DL models.
“However, large labelled datasets are required to train DL models (Fink et al., 2020).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Fink et al., 2020 source
Not checked in the free run.
- Premisecitation
DL algorithms are capable of learning more correct features compared to human-designed ones.
“On the other hand, DL algorithms are capable of learning more correct features compared to human-designed ones (Zhou et al., 2018).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Zhou et al., 2018 source
Not checked in the free run.
- Premisereasoning
ML model development practice differs from traditional software development due to its data dependency, uncertainty, and experimentation requirements.
“ML model development practice differs from traditional software development due to its data dependency, uncertainty, and experimentation requirements.”
Quote in the paperSubstantial claim - Premisecitation
Discovering, accessing, collecting, cleaning, and transforming data is challenging and time-consuming.
“Discovering, accessing, collecting, cleaning, and transforming data is challenging and time-consuming (Amershi et al., 2019;Baier et al., 2019;Correia et al., 2020;Fredriksson et al., 2020;Fujii et al., 2020;Hill et al., 2016;Kim et al., 2017;Lwakatare et al., 2019;Sankaran et al., 2017).”
Quote in the paperNot a substantial claim0 of 3 sources checked- cites Amershi et al., 2019
Not checked in the free run.
- cites Baier et al., 2019
Not checked in the free run.
- cites Correia et al., 2020
Not checked in the free run.
- Premisecitation
Customers generally do not understand the difference between traditional software and ML system.
“Customers generally do not understand the difference between traditional software and ML system (Baier et al., 2019;Vogelsang and Borg, 2019).”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites Baier et al., 2019
Not checked in the free run.
- cites Vogelsang and Borg, 2019
Not checked in the free run.
- Premisereasoning
Engineers do not have a mature set of solution patterns for solving the issues posed by ML.
“Engineers do not have a mature set of solution patterns for solving the issues posed by ML.”
Quote in the paperSubstantial claim - Premisecitation
ML systems pose new challenges for conflict resolution.
“ML systems pose new challenges for conflict resolution (Horkoff, 2019).”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Horkoff, 2019
Not checked in the free run.
- Premisecitation
It may be difficult to map these measures to business objectives.
“It may also be difficult to map these measures to business objectives (Baier et al., 2019;de Souza Nascimento et al., 2019).”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites Baier et al., 2019
Not checked in the free run.
- cites de Souza Nascimento et al., 2019
Not checked in the free run.
- Premisecitation
Customers face difficulties in selecting proper evaluation metrics and interpreting these for assessing the quality of an ML system.
“On the other hand, customers face difficulties in selecting proper evaluation metrics and interpreting these for assessing the quality of an ML system (Vogelsang and Borg, 2019).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Vogelsang and Borg, 2019
Not checked in the free run.
- Premisecitation
A degradation in how well an ML system meets users' expectations should be expected on production in time.
“A degradation in how well an ML system meets users' expectations should be expected on production in time (Wan et al., 2019).”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Wan et al., 2019
Not checked in the free run.
- Premisecitation
Requirements to be fulfilled by ML components are generally specified using quantitative measures (such as accuracy, precision, recall, F measure) that are new for many stakeholders.
“Requirements to be fulfilled by ML components are generally specified using quantitative measures (such as accuracy, precision, recall, F measure) (Kim et al., 2017;Vogelsang and Borg, 2019;Wan et al., 2019) that are new for many stakeholders.”
Quote in the paperNot a substantial claim0 of 3 sources checked- cites Kim et al., 2017 source
Not checked in the free run.
- cites Vogelsang and Borg, 2019
Not checked in the free run.
- cites Wan et al., 2019
Not checked in the free run.
- Premisecitation
This principle prevents the use of standard techniques (such as abstraction and information hiding) for reducing coupling.
“This principle prevents the use of standard techniques (such as abstraction and information hiding) for reducing coupling (Wan et al., 2019), which is a fundamental design principle for traditional software (Sculley et al., 2014).”
Quote in the paperSubstantial claim0 of 2 sources checked- cites Wan et al., 2019
Not checked in the free run.
- cites Sculley et al., 2014
Not checked in the free run.
- Premisecitation
The development of ML systems makes the RE process even more challenging.
“The development of ML systems makes the RE process even more challenging (Chechik, 2019;Dalpiaz and Niu, 2020).”
Quote in the paperSubstantial claim0 of 2 sources checked- cites Chechik, 2019
Not checked in the free run.
- cites Dalpiaz and Niu, 2020
Not checked in the free run.
- Premisecitation
In some domains, such as software analytics, making models explainable to users is as important as making accurate decisions.
“In some domains, such as software analytics, making models explainable to users is as important as making accurate decisions (Dam et al., 2018).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Dam et al., 2018
Not checked in the free run.
- Premisecitation
Data quality is an important aspect to be considered during RE since it directly affects the performance of an ML model.
“In addition, data quality is an important aspect to be considered during RE since it directly affects the performance of an ML model (Challa et al., 2020).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Challa et al., 2020
Not checked in the free run.
- Premisecitation
Working with a large volume of data that requires a distributed system brings additional complexity to development.
“Working with a large volume of data that requires a distributed system brings additional complexity to development (Arpteg et al., 2018).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Arpteg et al., 2018
Not checked in the free run.
- Premisecitation
Such integration may be challenging due to the different characteristics of ML and traditional software components.
“On the other hand, such integration may be challenging due to the different characteristics of ML and traditional software components (Amershi et al., 2019).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Amershi et al., 2019
Not checked in the free run.
- Premisecitation
Developers work on extracting and selecting informative features for models.
“Developers work on extracting selecting informative features for models (Amershi et al., 2019;Hill et al., 2016;Kim et al., 2017).”
Quote in the paperNot a substantial claim0 of 3 sources checked- cites Amershi et al., 2019
Not checked in the free run.
- cites Hill et al., 2016
Not checked in the free run.
- cites Kim et al., 2017 source
Not checked in the free run.
- Premisecitation
Developing ML models may require many iterations and experimentation, including feedback loops between sub-activities.
“Developing models may require many iterations and experimentation, including many feedback loops between sub-activities (Amershi et al., 2019;Pham et al., 2020).”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites Amershi et al., 2019
Not checked in the free run.
- cites Pham et al., 2020
Not checked in the free run.
- Premisecitation
This heterogeneity causes potential discrepancies among development, quality assurance, and production environments.
“This heterogeneity causes potential discrepancies among development, quality assurance, and production environments (Zhang et al., 2020a).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Zhang et al., 2020a
Not checked in the free run.
- Premisecitation
Tripakis (2018) and Menzies (2019) points out the heterogeneity of DL platforms that leads to interoperability problems.
“Tripakis (2018) and Menzies (2019) points out the heterogeneity of DL platforms that leads to interoperability problems.”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites Tripakis, 2018
Not checked in the free run.
- cites Menzies, 2019 source
Not checked in the free run.
- Interpretationreasoning
The difficulty of analyzing and understanding ML models may lead to problems in evaluating and debugging models.
“This difficulty may lead to problems in evaluating and debugging models.”
Quote in the paperNot a substantial claim - Premisecitation
ML model development is highly iterative.
“Due to the highly iterative nature of ML model development, the infrastructure should enable experiment management (Arpteg et al., 2018;Gharibi et al., 2019;Lwakatare et al., 2019).”
Quote in the paperSubstantial claim0 of 3 sources checked- cites Arpteg et al., 2018
Not checked in the free run.
- cites Gharibi et al., 2019
Not checked in the free run.
- cites Lwakatare et al., 2019
Not checked in the free run.
- Premisecitation
Testing ML systems poses challenges that arise from the fundamentally different nature and development of ML systems compared to relatively more deterministic traditional software systems.
“Testing ML systems poses challenges that arise from the fundamentally different nature and development of ML systems (as briefly explained in Section 2.2), compared to relatively more deterministic traditional software systems (Zhang et al., 2020c).”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Zhang et al., 2020c
Not checked in the free run.
- Premisecitation
Handling exceptions in data is a serious problem in developing ML models.
“Handling exceptions in data is a serious problem in developing ML models (Zhang et al., 2020a).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Zhang et al., 2020a
Not checked in the free run.
- Premisecitation
Many researchers mentioned the difficulty of test case generation for ML systems.
“Many researchers mentioned the difficulty of test case generation for ML systems (Ahmed et al., 2020;Chechik et al., 2019;Wan et al., 2019).”
Quote in the paperNot a substantial claim0 of 3 sources checked- cites Ahmed et al., 2020
Not checked in the free run.
- cites Chechik et al., 2019
Not checked in the free run.
- cites Wan et al., 2019
Not checked in the free run.
- Premisecitation
Changes in the ML model may negatively affect dependent components.
“In such cases, the changes in the ML model may negatively affect the dependent components (Belani et al., 2019;Hill et al., 2016;Lwakatare et al., 2019).”
Quote in the paperSubstantial claim0 of 3 sources checked- cites Belani et al., 2019
Not checked in the free run.
- cites Hill et al., 2016
Not checked in the free run.
- cites Lwakatare et al., 2019
Not checked in the free run.
- Rests on an assumption: The outputs of an ML model may be used by many other components implicitly.
- Premisecitation
Although there are some tools for exploring data dependencies, these are not as capable and common as static analysis tools for traditional software.
“Although there are some tools for exploring such data dependencies (McMahan et al., 2013), these are not as capable and common as the static analysis tools for traditional software.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites McMahan et al., 2013
Not checked in the free run.
- Premisecitation
A heterogeneous tool set challenges engineers in many tasks, such as data processing, environment setup, and model deployment.
“A heterogeneous tool set challenges engineers in many tasks, such as data processing, environment setup, and model deployment (Alshangiti et al., 2019).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Alshangiti et al., 2019
Not checked in the free run.
- Premisecitation
A frequently seen type of bug in ML systems is a 'data bug'.
“A frequently seen type of bug in ML systems is a”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Islam, 2019
Not checked in the free run.
- Premisecitation
Several ML models are vulnerable to adversarial examples, which are only slightly different from correctly classified examples drawn from the training data distribution.
“Several ML models are vulnerable to adversarial examples, which are only slightly different from correctly classified examples drawn from the training data distribution (Szegedy et al., 2014).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Szegedy et al., 2014
Not checked in the free run.
- Premisecitation
The presence of diverse and incompatible programming and data tools is a severe concern for engineering ML systems.
“The presence of diverse and incompatible programming and data tools is a severe concern for engineering ML systems (Kim et al., 2017).”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Kim et al., 2017 source
Not checked in the free run.
- Premisecitation
In some cases, a lot of manual work may be needed for deployment.
“In some cases, a lot of manual work may be needed for deployment (Baier et al., 2019).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Baier et al., 2019
Not checked in the free run.
- Premisecitation
ML components generally depend on external data.
“On the other hand, ML components generally depend on external data (Sculley et al., 2015).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Sculley et al., 2015
Not checked in the free run.
- Premisecitation
One reason for the difficulty of debugging ML systems is the prevalence of non-determinism in the training process, which leads to hard-to-reproduce bugs.
“One reason for this is the prevalence of non-determinism in the training process, which leads to hard-toreproduce bugs (Wan et al., 2019;Zhang et al., 2018b).”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites Wan et al., 2019
Not checked in the free run.
- cites Zhang et al., 2018b
Not checked in the free run.
- Interpretationreasoning
All of the reviewed approaches for selecting test subsets try to reduce input data labeling and testing effort.
“All of these approaches try to reduce input data labeling and testing effort.”
Quote in the paperNot a substantial claim - Premisecitation
Adversarial examples cause ML models to produce an error output with high confidence, even though their difference from natural inputs are subtle.
“Adversarial examples cause ML models to produce an error output with high confidence, even though their difference from natural inputs are subtle (Szegedy et al., 2014).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Szegedy et al., 2014
Not checked in the free run.
- Premisecitation
Some of the generated images (test inputs) were still unnatural and could not be recognized by humans.
“On the other hand, some of the generated images (test inputs) were still unnatural and could not be recognized by humans (Zhang et al., 2020c).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Zhang et al., 2020c
Not checked in the free run.
- Premisecitation
In some cases, collecting testing datasets may require manual labeling, which is labor-intensive.
“In some cases, collecting testing datasets may require manual labeling, which is labor-intensive (Li et al., 2019b;Wan et al., 2019).”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites Li et al., 2019b
Not checked in the free run.
- cites Wan et al., 2019
Not checked in the free run.
- Premisecitation
ML systems heavily depend on ML libraries, frameworks, and platforms.
“ML systems heavily depend on ML libraries, frameworks, and platforms.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Guo et al., 2019
Not checked in the free run.
- Premisecitation
Fairness can be an essential quality attribute of ML systems.
“Fairness can be an essential quality attribute of ML systems (Corbett-Davies and Goel, 2018), as pointed out in Section 5.3.1.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Corbett-Davies and Goel, 2018
Not checked in the free run.
- Premisecitation
Unlike traditional software, code coverage cannot be used for ML components since the decision logic of ML capability is obtained via training an ML model, not via explicit coding.
“Unlike traditional software, code coverage cannot be used for ML components since the decision logic of ML capability is obtained via training an ML model, not via explicit coding (Ishikawa and Yoshioka, 2019;Khomh et al., 2018;Kim et al., 2019;Ma et al., 2018b;Sekhon and Fleming, 2019;Sun et al., 2018;Zhang et al., 2020c).”
Quote in the paperNot a substantial claim0 of 3 sources checked- cites Ishikawa and Yoshioka, 2019
Not checked in the free run.
- cites Khomh et al., 2018 source
Not checked in the free run.
- cites Kim et al., 2019
Not checked in the free run.
- Premisecitation
The non-deterministic nature of ML systems makes test evaluation more challenging.
“The non-deterministic nature of ML systems makes test evaluation more challenging (Arpteg et al., 2018;Ishikawa, 2018;Ishikawa and Yoshioka, 2019;Nishi et al., 2018;Tao et al., 2019).”
Quote in the paperSubstantial claim0 of 3 sources checked- cites Arpteg et al., 2018
Not checked in the free run.
- cites Ishikawa, 2018
Not checked in the free run.
- cites Ishikawa and Yoshioka, 2019
Not checked in the free run.
- Premisecitation
The bugs in ML libraries, frameworks, and platforms impair ML systems.
“The bugs in these components impair ML systems (Guo et al., 2019).”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Guo et al., 2019
Not checked in the free run.
- Premisecitation
Good performance during testing cannot guarantee the satisfactory performance of ML systems on production.
“Moreover, good performance during testing cannot guarantee the satisfactory performance of ML systems on production (Wan et al., 2019).”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Wan et al., 2019
Not checked in the free run.
- Premisecitation
Access to high quality test data is a concern for developing ML systems.
“Access to high quality test data is a concern for developing ML systems (Arpteg et al., 2018;Guo et al., 2018;Knauss et al., 2017;Li et al., 2019b;Ma et al., 2018a;Ma et al., 2018b;Wan et al., 2019).”
Quote in the paperNot a substantial claim0 of 3 sources checked- cites Arpteg et al., 2018
Not checked in the free run.
- cites Guo et al., 2018
Not checked in the free run.
- cites Knauss et al., 2017
Not checked in the free run.
- Premisecitation
Unlike testing traditional software, finding one or a few incorrect results does not necessarily indicate the presence of a bug.
“Unlike testing traditional software, finding one or a few incorrect results does not necessarily indicate the presence of a bug (Dwarakanath et al., 2018).”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Dwarakanath et al., 2018
Not checked in the free run.
- Premisecitation
Garcia et al. identified 499 bugs in Apollo Baidu and Autoware.
“Garcia et al. (2020) identified 499 bugs in these two software systems.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Garcia et al., 2020
Not checked in the free run.
- Premisecitation
Test case evaluation is a complex task since it is hard to formalize and measure test cases' characteristics influence quality.
“Test case evaluation is a complex task since it is hard to formalize and measure test cases' characteristics influence quality (Durelli et al., 2019).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Durelli et al., 2019 source
Not checked in the free run.
- Premisecitation
The difficulties of debugging ML systems were pointed out by several studies.
“The difficulties of debugging ML systems were pointed out by several studies (Khomh et al., 2018;Wan et al., 2019;Zhang et al., 2018b;Zhang et al., 2019a).”
Quote in the paperNot a substantial claim0 of 3 sources checked- cites Khomh et al., 2018 source
Not checked in the free run.
- cites Wan et al., 2019
Not checked in the free run.
- cites Zhang et al., 2018b
Not checked in the free run.
- Premisecitation
Another pain point is the lack of debugging and profiling support for ML systems.
“Another pain point is the lack of debugging and profiling support for ML systems (Zhang et al., 2019a;Zhang et al., 2020a).”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites Zhang et al., 2019a
Not checked in the free run.
- cites Zhang et al., 2020a
Not checked in the free run.
- Premisecitation
Data lifecycle activities form a critical and compelling subgroup of activities, which profoundly affect ML projects' success.
“Data lifecycle activities form a critical and compelling subgroup of activities, which profoundly affect ML projects' success (Munappy et al., 2020;Polyzotis et al., 2018;Wan et al., 2019).”
Quote in the paperSubstantial claim0 of 3 sources checked- cites Munappy et al., 2020
Not checked in the free run.
- cites Polyzotis et al., 2018 source
Not checked in the free run.
- cites Wan et al., 2019
Not checked in the free run.
- Premisecitation
Software Composition Analysis (SCA) tools can help in discovering all related components, their supporting libraries, and their direct and indirect dependencies.
“Software Composition Analysis (SCA) tools can help in discovering all related components, their supporting libraries, and their direct and indirect dependencies (Han et al., 2020).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Han et al., 2020
Not checked in the free run.
- Premisecitation
Another critical maintenance issue is the technical debts present in ML frameworks.
“Another critical maintenance issue is the technical debts present in ML frameworks (Liu et al., 2020b).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Liu et al., 2020b
Not checked in the free run.
- Premisecitation
Automatic maintenance mechanisms can be built to keep the performance at a certain level.
“Automatic maintenance mechanisms can be built to keep the performance at a certain level (Wan et al., 2019).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Wan et al., 2019
Not checked in the free run.
- Premisecitation
Liu et al. identified design, defect, documentation, test, requirement, compatibility, and algorithm debts in ML frameworks, which may cause maintenance problems in ML systems.
“Liu et al. (2020b) identified design, defect, documentation, test, requirement, compatibility, and algorithm debts in ML frameworks, which may cause maintenance problems in ML systems.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Liu et al., 2020b
Not checked in the free run.
- Premisecitation
Unforeseen changes in the external world may cause changes in input data patterns and negatively affect the performance of ML components (concept drift).
“Unforeseen changes in the external world may cause changes in input data patterns and negatively affect the performance of ML components (known as “concept drift” (Tsymbal, 2004), see Section 5.3.2).”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Tsymbal, 2004
Not checked in the free run.
- Premisecitation
Data and ML models are essential configuration items besides code.
“Data and ML models are essential configuration items besides code (Belani et al., 2019;Sato et al., 2019).”
Quote in the paperSubstantial claim0 of 2 sources checked- cites Belani et al., 2019
Not checked in the free run.
- cites Sato et al., 2019
Not checked in the free run.
- Premisecitation
It is prevalent to perform a vast number of experiments to identify the optimal ML model.
“The development of ML components is an iterative and experimental process (Gharibi et al., 2019), and hence, it is prevalent to perform a vast number of experiments to identify the optimal ML model (Arpteg et al., 2018).”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Arpteg et al., 2018
Not checked in the free run.
- Premisecitation
Maintaining ML systems involves keeping track of additional configuration items, models, and data, besides code.
“Maintaining ML systems involves keeping track of additional configuration items, models, and data, besides code (Sato et al., 2019).”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Sato et al., 2019
Not checked in the free run.
- Premisecitation
The development of ML components is an iterative and experimental process.
“The development of ML components is an iterative and experimental process (Gharibi et al., 2019), and hence, it is prevalent to perform a vast number of experiments to identify the optimal ML model (Arpteg et al., 2018).”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Gharibi et al., 2019
Not checked in the free run.
- Premisecitation
Improper experiment management may result in some experiments being repeated unintentionally.
“( 2) some experiments may be repeated unintentionally (Hill et al., 2016);”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Hill et al., 2016
Not checked in the free run.
- Premisecitation
It is a challenge to keep pace with the latest versions of ML libraries and their dependencies.
“It is a challenge to keep pace with the latest versions of ML libraries and their dependencies (Han et al., 2020).”
- Also stated in: § 6.1 POTENTIAL RESEARCH DIRECTIONS FROM THE SE PERSPECTIVE · plain
“Keeping pace with the latest versions of ML libraries and their dependencies is a challenge (Han et al., 2020).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Han et al., 2020
Not checked in the free run.
- Rests on an assumption: ML models are dependent on ML frameworks.
- Also stated in: § 6.1 POTENTIAL RESEARCH DIRECTIONS FROM THE SE PERSPECTIVE · plain
- Premisecitation
ML model lifecycle activities are another vital subgroup of activities integrated with the SE process.
“ML model lifecycle activities are another vital subgroup of activities integrated with the SE process (Alvarez-Rodríguez et al., 2019;Byrne, 2017).”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites Alvarez-Rodríguez et al., 2019
Not checked in the free run.
- cites Byrne, 2017
Not checked in the free run.
- Premisecitation
Improper experiment management may result in experiments not being reproduced when needed.
“(1) experiments may not be reproduced when needed (Hill et al., 2016;Zaharia et al., 2018);”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites Hill et al., 2016
Not checked in the free run.
- cites Zaharia et al., 2018
Not checked in the free run.
- Premisecitation
Model management has been identified as one of the most challenging and time-consuming activities.
“Model management, which refers to tracking, storing, query, comparing, reproducing, and sharing models (Gharibi et al., 2019), has been identified as one of the most challenging and time-consuming activities (Schelter et al., 2018;Vartak, 2018).”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites Schelter et al., 2018
Not checked in the free run.
- cites Vartak, 2018
Not checked in the free run.
- Premisecitation
Teams may require to compare, reproduce, and share experiments.
“Teams may require to compare, reproduce, and share experiments (Gharibi et al., 2019;Lwakatare et al., 2019).”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites Gharibi et al., 2019
Not checked in the free run.
- cites Lwakatare et al., 2019
Not checked in the free run.
- Premisecitation
Another challenge arises when organizations decide to enhance their existing systems with ML capabilities.
“Another challenge arises when organizations decide to enhance their existing systems with ML capabilities (Jahić and Roitsch, 2020).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Jahić and Roitsch, 2020
Not checked in the free run.
- Interpretationown data
This trend clearly shows the increasing interest of the SE community in engineering ML systems.
“This trend clearly shows the increasing interest of the SE community in engineering ML systems.”
Quote in the paperNot a substantial claim - Premisecitation
Part of the organizational challenges involves identifying skill sets and roles required to engineer ML systems.
“Part of the organizational challenges involves identifying skill sets and roles required to engineer ML systems (Flaounas, 2017).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Flaounas, 2017
Not checked in the free run.
- Premisecitation
As a result of not being able to estimate effort, business owners act impatiently and cancel ML component development despite promising intermediate results.
“As a result of not being able to estimate effort, business owners act impatiently and cancel ML component development despite promising intermediate results (Arpteg et al., 2018).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Arpteg et al., 2018
Not checked in the free run.
- Premisecitation
It is even challenging to have all these people on the same page.
“It is even challenging to have all these people on the same page (Kim et al., 2017).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Kim et al., 2017 source
Not checked in the free run.
- Premisecitation
Lack of developer support tools for data-intensive systems is a challenge for developing ML systems.
“Lack of developer support tools for data-intensive systems is a challenge for developing ML systems (Anderson, 2015).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Anderson, 2015
Not checked in the free run.
- Premisecitation
Engineering an ML system involves skill sets from multiple fields, i.e., machine learning, deep learning, data science, mathematics, algorithms, software engineering, and statistics.
“Engineering an ML system involves skill sets from multiple fields, i.e., machine learning, deep learning, data science, mathematics, algorithms, software engineering, and statistics (Alshangiti et al., 2019;Alvarez-Rodríguez et al., 2019;Belani et al., 2019;Correia et al., 2020;Wan et al., 2019;Workera, 2020).”
- Also stated in: § 6.1 POTENTIAL RESEARCH DIRECTIONS FROM THE SE PERSPECTIVE · plain
“Engineering an ML system involves skill sets from various fields, i.e., machine learning, deep learning, data science, mathematics, algorithms, software engineering, and statistics (Alshangiti et al., 2019;Alvarez-Rodríguez et al., 2019;Belani et al., 2019;Wan et al., 2019;Workera, 2020).”
Quote in the paperNot a substantial claim0 of 3 sources checked- cites Alshangiti et al., 2019
Not checked in the free run.
- cites Alvarez-Rodríguez et al., 2019
Not checked in the free run.
- cites Belani et al., 2019
Not checked in the free run.
- Also stated in: § 6.1 POTENTIAL RESEARCH DIRECTIONS FROM THE SE PERSPECTIVE · plain
- Premisecitation
In some cases, the team cannot know whether the quantitative targets are achievable until final ML models are obtained.
“In some cases, the team cannot know whether the quantitative targets are achievable until final ML models are obtained (Wan et al., 2019).”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Wan et al., 2019
Not checked in the free run.
- Premisecitation
The synergy between data scientists, domain experts, and customers during data processing is essential for proper data collection, data enrichment and labeling.
“For instance, the synergy between data scientists, domain experts, and customers during data processing is essential for proper data collection, data enrichment and labeling (Correia et al., 2020).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Correia et al., 2020
Not checked in the free run.
- Premisecitation
Some tools emerged from academia, such as DARVIZ, DeepVisual, NeuralVis, ModelKB, to solve specific problems in developing ML systems.
“Some tools emerged from academia, such as DARVIZ (Sankaran et al., 2017), DeepVisual (Xie et al., 2019a), NeuralVis (Zhang et al., 2019b), ModelKB (Gharibi et al., 2019), to solve specific problems in developing ML systems.”
Quote in the paperNot a substantial claim0 of 3 sources checked- cites Sankaran et al., 2017
Not checked in the free run.
- cites Xie et al., 2019a
Not checked in the free run.
- cites Zhang et al., 2019b
Not checked in the free run.
- Premisecitation
A proliferation of roles, including specialists, in particular SE phases, such as design or specific types of algorithms such as natural language processing.
“A proliferation of roles, including specialists, in particular, SE phases, such as design (Northrop et al., 2019) or specific types of algorithms such as natural language processing (Byrne, 2017).”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites Northrop et al., 2019 source
Not checked in the free run.
- cites Byrne, 2017
Not checked in the free run.
- Premisecitation
Practitioners and academicians already started to identify emerging roles to classify skill sets and have proper people in teams to engineer ML systems.
“Practitioners and academicians already started to identify emerging roles to classify skill sets and have proper people in teams to engineer ML systems (Kim et al., 2017;Workera, 2020).”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites Kim et al., 2017 source
Not checked in the free run.
- cites Workera, 2020
Not checked in the free run.
- Premisecitation
It is difficult and mostly impossible to characterize all the behaviors of ML components under all circumstances.
“It is difficult and mostly impossible to characterize all the behaviors of ML components under all circumstances (Tuncali et al., 2019).”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Tuncali et al., 2019 source
Not checked in the free run.
- Premisecitation
It is challenging to get all these people working together towards achieving a common objective.
“Therefore, it is challenging to get all these people working together towards achieving a common objective (Amershi et al., 2019).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Amershi et al., 2019
Not checked in the free run.
- Premisecitation
To support other SE aspects, such as generating Python components ready for deployment, version control conflict handling, and continuous integration, some tools, such as nbdev, have been released.
“To support other SE aspects, such as generating Python components ready for deployment, version control conflict handling, and continuous integration, some tools, such as nbdev, have been released (Howard and Gugger, 2020).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Howard and Gugger, 2020
Not checked in the free run.
- Interpretationnone given
Despite many studies, we do not have mature testing techniques and tools to develop ML systems in industrial settings.
“Despite many studies, we do not have mature testing techniques and tools to develop ML systems in industrial settings.”
Quote in the paperNot a substantial claim - Premisecitation
There are some platforms, such as MLflow, focusing only on the ML lifecycle.
“There are some platforms, such as MLflow (Zaharia et al., 2018), focusing only on the ML lifecycle.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Zaharia et al., 2018
Not checked in the free run.
- Premisecitation
Open-source ML libraries include many bugs.
“Open-source ML libraries include many bugs (Garcia et al., Pham et al., 2019;2020;Wang et al., 2020b).”
Quote in the paperNot a substantial claim0 of 3 sources checked- cites Garcia et al.
Not checked in the free run.
- cites Pham et al., 2019
Not checked in the free run.
- cites Wang et al., 2020b
Not checked in the free run.
- Study design and conductreasoning
Selection of primary studies only from SE venues forms a threat to having a complete set of primary studies.
“On the other hand, selection of primary studies only from SE venues forms a threat to having a complete set of primary studies.”
Quote in the paperNot a substantial claim - Premisecitation
Debugging and profiling tool support is an essential deficiency for engineering ML systems.
“Debugging and profiling tool support is an essential deficiency for engineering ML systems (Zhang et al., 2019a).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Zhang et al., 2019a
Not checked in the free run.
- Premisecitation
Coverage criteria designed for ML/DL models have been used to evaluate test cases.
“Coverage criteria designed for ML/DL models have been used to evaluate test cases (Ma et al., 2018a;Sun et al., 2018;Sun et al., 2019b;Du et al., 2019).”
Quote in the paperNot a substantial claim0 of 3 sources checked- cites Ma et al., 2018a
Not checked in the free run.
- cites Sun et al., 2018
Not checked in the free run.
- cites Sun et al., 2019b
Not checked in the free run.
- Interpretationreasoning
Compared to tool support for traditional software development, there is room for improvement to have SE tooling for developing ML systems.
“On the other hand, compared to tool support for traditional software development, there is room for improvement to have SE tooling for developing ML systems.”
Quote in the paperSubstantial claim - Premisecitation
Mutation testing has been used to address the shortcomings of coverage criteria.
“Mutation testing (MuT) has been used to address the shortcomings of coverage criteria (Jia and Harman, 2010;Zhang et al., 2018c).”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites Jia and Harman, 2010
Not checked in the free run.
- cites Zhang et al., 2018c
Not checked in the free run.
- Premisecitation
Another maintainability issue related to ML libraries is the technical debt present in these libraries.
“Another maintainability issue related to ML libraries is the technical debt present in these libraries (Liu et al., 2020b).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Liu et al., 2020b
Not checked in the free run.
- Premisecitation
As in testing scientific software, perfect oracles may not be available for ML systems.
“As in testing scientific software (Kanewala and Bieman, 2014), perfect oracles may not be available for ML systems.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Kanewala and Bieman, 2014 source
Not checked in the free run.
- Premisecitation
There is empirical evidence showing that single data extraction results in more errors than double data extraction.
“For instance, there is empirical evidence showing that single data extraction results in more errors than double data extraction (Buscemi et al., 2006).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Buscemi et al., 2006 source
Not checked in the free run.
- May combine two claims. 62
- Classified as probably not a substantial claim. 155
- The quote only partly states this claim. 1
May combine two claims.
Findingnone givenNone of the existing reviews addressing SE aspects cover papers published in 2020, while this study covers papers published until the end of 2020 in SE venues (75 papers in total).
“While none of the existing reviews addressing SE aspects (Kumeno, 2019;Lwakatare et al., 2020;Nascimento et al., 2020) cover the papers published in 2020, this study covers the papers published until the end of 2020 in SE venues (75 papers in total).”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Kumeno, 2019;Lwakatare et al., 2020;Nascimento et al., 2020
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
FindingreasoningThe total number of primary studies equals 147 in Figure 6.
“Due to these three primary studies using multiple research methods, the total number of primary studies equals 147 in Figure 6.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Findingown dataThematic analysis, case study, and statistical analysis are the other approaches used by the researchers.
“Thematic analysis, case study, and statistical analysis are the other approaches used by the researchers.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
FindingcitationFour studies mention ML systems predicting values for combined sewer overflow, stock price, age, real estate value and weather.
“Four studies mention ML systems predicting values for combined sewer overflow (Challa et al., 2020), stock price (Wang et al., 2020b), age (Gao et al., 2020a), real estate value and weather (Arpteg et al., 2018).”
Quote in the paperNot a substantial claim0 of 3 sources checked- cites Challa et al., 2020
Not checked in the free run.
- cites Wang et al., 2020b
Not checked in the free run.
- cites Gao et al., 2020a
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
Findingown dataOne case study (Nakamichi et al., 2020) addressing requirements engineering challenges did not report its application scenario(s).
“One case study (Nakamichi et al., 2020) addressing requirements engineering challenges did not report its application scenario(s).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Nakamichi et al., 2020
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
Findingown dataFigure 8 shows the distribution of application scenarios used in 93 of the primary studies, the research method of which is either an experiment or a case study.
“Figure 8 shows the distribution of application scenarios used in 93 of the primary studies, the research method of which is either an experiment or a case study.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
FindingcitationOne of Gao et al.'s experiments involved BERT model over GLUE benchmark, with various batch sizes and sequence lengths.
“One of their experiments involved BERT (Devlin et al., 2018) model over GLUE (General Language Understanding Evaluation) benchmark (Wang et al., 2018), with various batch sizes and sequence lengths.”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites Devlin et al., 2018
Not checked in the free run.
- cites Wang et al., 2018
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
FindingcitationLiu et al. (2020b) explored technical debts in seven open-source DL frameworks and their impacts on the maintainability of DL systems.
“Liu et al. (2020b) explored technical debts in seven open-source DL frameworks and their impacts on the maintainability of DL systems.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Liu et al., 2020b
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
Findingnone givenThree studies involved an ML system for object detection.
“Three studies involved an ML system for object detection.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingown dataBesides experiments, the researchers interviewed and surveyed experts to learn about the challenges they face in engineering ML systems.
“Besides experiments, the researchers interviewed and surveyed experts to learn about the challenges they face in engineering ML systems.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
FindingcitationTizpaz-Niari et al. (2020) discovered performance bugs in ML frameworks, such as the implementation of logistic regression in scikit-learn.
“Tizpaz-Niari et al. (2020) discovered performance bugs in ML frameworks, such as the implementation of logistic regression in scikit-learn.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Tizpaz-Niari et al., 2020
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
Findingown dataOnly one study (Murphy et al., 2007) is concerned with a ranking problem.
“Only one study (Murphy et al., 2007) is concerned with a ranking problem.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Murphy et al., 2007
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
Findingown dataTwo other studies (Humbatova et al., 2020; Zhang et al., 2020a) used thematic analysis and interview methods together.
“Two other studies (Humbatova et al., 2020;Zhang et al., 2020a) used thematic analysis and interview methods together.”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites Humbatova et al., 2020
Not checked in the free run.
- cites Zhang et al., 2020a
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
FindingcitationSun and Zhou (2018) and Zheng et al. (2019) addressed the challenges of testing an ML system on machine translation.
“Sun and Zhou (2018) and Zheng et al. (2019) addressed the challenges of testing an ML system on machine translation.”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites Sun and Zhou, 2018
Not checked in the free run.
- cites Zheng et al., 2019
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
Findingown dataThe terms representing the intervention of the study (machine learning or deep learning) include 'Deep Learning,' 'Machine Learning,' 'learning,' 'model,' 'Deep Neural Network,' 'Artificial Intelligence,' 'training,' 'DL model,' 'neural network,' and 'ML model'.
“There are also many terms representing the intervention of this study, i.e.,”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
FindingcitationThe included studies were divided into empirical (127 papers) and non-empirical (15 papers).
“I divided the included studies into empirical (127 papers) and non-empirical (15 papers), as suggested by Mohanani et al. (2018).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Mohanani et al. (2018)
Not checked in the free run.
May combine two claims.
Findingnone givenAhmed et al. (2020), Dutta et al. (2020), and Wang et al. (2020b) studied problems in DL/ML frameworks related to testing and quality.
“Ahmed et al. (2020), Dutta et al. (2020), and Wang et al. (2020b) studied problems in DL/ML frameworks related to testing and quality.”
Quote in the paperNot a substantial claim0 of 3 sources checked- cites Ahmed et al., 2020
Not checked in the free run.
- cites Dutta et al., 2020
Not checked in the free run.
- cites Wang et al., 2020b
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
Findingown dataTwo studies (Amershi et al., 2019; Wan et al., 2019) combined interview and survey methods.
“Two studies (Amershi et al., 2019;Wan et al., 2019) combined interview and survey methods.”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites Amershi et al., 2019
Not checked in the free run.
- cites Wan et al., 2019
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
Findingown dataThe terms representing the population of the study (software engineering) include 'software engineering,' 'development,' 'developer,' 'testing,' 'test,' 'bug,' 'quality,' 'coverage,' 'criterion,' 'requirement,' 'performance,' 'robustness,' 'safety,' 'analysis,' and 'failure'.
“We can observe many terms that represent the population of this study, i.e.,”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
FindingcitationAlshangiti et al. (2019) performed thematic and statistical analysis together.
“Alshangiti et al. (2019) performed thematic and statistical analysis together.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Alshangiti et al., 2019
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
FindingcitationMurphy et al. (2007) aim at ranking the probabilities of observing a failure in a device.
“Murphy et al. (2007) aim at ranking the probabilities of observing a failure in a device.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Murphy et al., 2007
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
FindingcitationPham et al. (2020) conducted experiments to study the variance of DL systems and a survey to understand the awareness of this variance among researchers and practitioners.
“Pham et al. (2020) conducted experiments to study the variance of DL systems and a survey to understand the awareness of this variance among researchers and practitioners.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Pham et al., 2020
Not checked in the free run.
May combine two claims.
Findingown dataThree studies contained application scenarios of Natural Language Processing, two specifically machine translation.
“Three studies contained application scenarios of Natural Language Processing, two specifically machine translation.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingnone givenFifteen papers either provide the author's opinions or did not report enough details about the research method.
“Fifteen papers either provide the author's opinions or did not report enough details about the research method.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
FindingcitationTrujillo et al. (2020) used mountain car problem to investigate whether neuron coverage can be used for deep reinforcement learning systems.
“Trujillo et al. (2020) used mountain car problem to investigate whether neuron coverage can be used for deep reinforcement learning systems.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Trujillo et al., 2020
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
Findingown dataTable 7 shows the list of datasets used more than once in the experiments and case studies in 93 primary studies.
“Table 7 shows the list of datasets used more than once in the experiments and case studies in 93 primary studies.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
FindingcitationGao et al. (2020b) proposed an estimation tool for GPU memory consumption of DL models to help developers in planning runtime resource requirements.
“Gao et al. (2020b) proposed an estimation tool for GPU memory consumption of DL models to help developers in planning runtime resource requirements.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Gao et al., 2020b
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
Findingown dataThe terms 'challenge' and 'problem' are among the most used terms by the authors to refer to the difficulties they face.
“Finally yet importantly,”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingown dataThe most frequently used 50 terms in the primary studies' abstracts include 'system,' 'software,' 'application,' 'DL software,' 'ML system,' and 'program'.
“Not surprisingly,”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
FindingcitationSculley et al. (2014) reported one of the complexities posed by ML systems as the 'change anything, changes everything' principle, which refers to the dependencies among all the parts of an ML system, i.e., application code, 'glue code,' ML libraries, and external data.
“They reported one of the complexities posed by ML systems as the”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Sculley et al., 2014
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
FindingcitationWashizaki et al. (2019b) state that ad-hoc solutions are being used for solving everyday problems in developing ML systems.
“Washizaki et al. (2019b) state that ad-hoc solutions are being used for solving everyday problems in developing ML systems.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Washizaki et al., 2019b
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
FindingcitationWan et al. (2019) interviewed professionals with experience in both traditional and ML system development and reported that nearly every interviewee made a strong statement about differences between the requirements of ML versus non-ML systems.
“Wan et al. (2019) interviewed professionals with experience in both traditional and ML system development and reported that nearly every interviewee made a strong statement about differences between the requirements of ML versus non-ML systems.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Wan et al., 2019
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
FindingcitationBenton et al. (2020) proposes an architecture for an ML system that deals with high-volume data, including a component for data federation to deal with structured, unstructured, and streaming data.
“Benton et al. (2020) proposes an architecture for an ML system that deals with high-volume data.This architecture includes a component for data federation to deal with structured, unstructured, and streaming data.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Benton et al., 2020 source
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
FindingcitationJiarpakdee et al. (2020) argued that the lack of explainability of defect prediction models could hinder the adoption of these models in practice.
“For instance, Jiarpakdee et al. (2020) argued that the lack of explainability of defect prediction models could hinder the adoption of these models in practice.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Jiarpakdee et al., 2020
Not checked in the free run.
May combine two claims.
Findingreasoning184 datasets were reported in 93 studies, which means most of the studies used two datasets on average.
“184 datasets were reported in 93 studies, which means most of the studies used two datasets on average.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
FindingcitationJohn et al. (2020a; 2020b) proposed a framework involving five architectural alternatives ranging from centralized cloud to fully decentralized edge architectures for designing ML systems.
“John et al. (2020a; 2020b) proposed a framework involving five architectural alternatives ranging from centralized cloud to fully decentralized edge architectures for designing ML systems.”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites John et al., 2020a
Not checked in the free run.
- cites John et al., 2020b
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
FindingcitationYokoyama (2019) proposes an architectural pattern for locating performance problems and rolling back in case of a failure on production.
“Yokoyama (2019) proposes an architectural pattern for locating performance problems and rolling back in case of a failure on production.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Yokoyama, 2019
Not checked in the free run.
Probably not a substantial claim: may not pass S1 (asserted), S3 (load-bearing). May combine two claims.
FindingcitationWashizaki et al. (2020) identified 15 design patterns for ML systems and plan to expand the list via further empirical studies.
“Washizaki et al. (2020) identified 15 design patterns for ML systems and plan to expand the list via further empirical studies.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Washizaki et al., 2020
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
FindingcitationHashemi et al. (2020) automatically generated ML system documentation from StackOverflow Q&As to provide more comprehensive knowledge at different level of expertise.
“To provide more comprehensive knowledge at different level of expertise, Hashemi et al. (2020) automatically generated ML system documentation from StackOverflow Q&As.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Hashemi et al., 2020
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
Findingown dataThe paper classifies the challenges in testing ML systems into eight categories.
“I classified these challenges into eight categories.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing), S4 (based).
Findingnone givenThe top SE conferences, i.e., ESEC/FSE and ICSE, receive submissions on this emerging topic.
“In addition, the top SE conferences, i.e., ESEC/FSE and ICSE, receive submissions on this emerging topic.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing), S4 (based).
Findingnone givenThe number of primary studies in SE venues on engineering ML systems has approximately doubled annually every year since 2018.
“The number of primary studies in SE venues on engineering ML systems has approximately doubled annually every year since 2018.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataOne hundred and one papers (72%) were authored solely by academic researchers, 11 papers (8%) by researchers from research organizations or companies, and 29 papers (20%) were written jointly by universities and other organizations.
“One hundred and one papers (72%) were authored solely by academic researchers, 11 papers (8%) by researchers from research organizations or companies, and 29 papers (20%) were written jointly by universities and other organizations.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataDevelopment & tools and requirements engineering areas are the second and third areas in which researchers collaborated.
“Development & tools and requirements engineering areas the second and third areas in which researchers collaborated.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Findingown dataThree papers were published in 2017, after which we see a sharp increase starting from 2018.
“Three papers were published in 2017, after which we see a sharp increase starting from 2018.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingown dataResearchers collaborated the most within the scope of testing and quality.
“Also, researchers collaborated the most within the scope of testing and quality.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingown dataThe earliest paper (Murphy et al., 2007) was published at the International Conference on Software Engineering & Knowledge Engineering in 2007.
“The earliest paper (Murphy et al., 2007) was published at the International Conference on Software Engineering & Knowledge Engineering in 2007.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Murphy et al., 2007
Not checked in the free run.
Probably not a substantial claim: may not pass S4 (based).
Findingnone givenRegression testing of ML systems has not been addressed in the primary studies.
“Regression testing of ML systems has not been addressed in the primary studies.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing), S4 (based).
Findingnone givenSE researchers focused mostly on testing ML systems.
“SE researchers focused mostly on testing ML systems.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing), S4 (based).
Findingnone givenPlanning, controlling, and monitoring of testing activities for ML systems have not been addressed by researchers.
“Planning, controlling, and monitoring of testing activities for ML systems have not been addressed by researchers.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingown dataThe conference with the most papers (20) is the ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE).
“The conference with the most papers (20) is the ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE).”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingown dataResearchers have been showing an increasing interest in this research area since 2018.
“Researchers have been showing an increasing interest in this research area since 2018 (Figure 9).”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingnone givenCase studies enabled researchers to collect data on activities related to ML systems in the industry.
“Case studies enabled researchers to collect data on activities related to ML systems in the industry.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingown dataExperts mentioned many challenges covering all aspects of engineering ML systems in interviews and surveys.
“As expected, experts mentioned many challenges covering all aspects of engineering ML systems in interviews and surveys.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Findingown dataThe DB search yielded 90 results, snowballing 68, and manual search 58.
“As shown in Figure4, the DB search yielded 90 results, snowballing 68, and manual search 58.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingown dataThe journals with the most papers (three in each) are IEEE Access, IEEE Software, and IEEE Transactions on Software Engineering.
“The journals with the most papers (three in each) are IEEE Access, IEEE Software, and IEEE Transactions on Software Engineering.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingown data91% of the studies were presented in conferences and workshops (129 papers in conferences/workshops and 12 papers in journals).
“91% of the studies were presented in conferences and workshops (129 papers in conferences/workshops and 12 papers in journals).”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataNineteen papers used traditional ML algorithms, and four papers used both DL and traditional ML algorithms.
“Nineteen papers used traditional ML algorithms, and four papers used both DL and traditional ML algorithms.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataAlthough there are a lot of primary studies and secondary studies on this topic, many research questions remain unanswered.
“Although there are a lot of primary studies (Section 9.1) and secondary studies (Section 3) on this topic, many research questions remain unanswered (Section 6.1 and Table 8).”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Findingown dataThere were 77 studies found via only one search method (DB search: 40, Snowballing: 31, Manual search: 6) and 64 studies via more than search method.
“There were 77 studies found via only one search method (DB search: 40, Snowballing: 31, Manual search: 6) and 64 studies via more than search method.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
PremisecitationMany researchers from SE and ML, as well as industry practitioners, have stated the requirement of a holistic view of engineering ML systems.
“Many researchers from software engineering (SE) (Hesenius et al., 2019;Northrop et al., 2019) and ML (Karpathy, 2017;Menzies, 2019), as well as industry practitioners (Benton, 2020;Booch, 2019;Lin and Kolcz, 2012) have stated the requirement of such a holistic view.”
Quote in the paperSubstantial claim0 of 3 sources checked- cites Hesenius et al., 2019;Northrop et al., 2019
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites Karpathy, 2017;Menzies, 2019
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites Benton, 2020;Booch, 2019;Lin and Kolcz, 2012
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductcitationThe research method was based on established guidelines.
“The research method was based on established guidelines (Kitchenham and Charters, 2007;Wohlin, 2014), some previous good examples (Durelli et al., 2019;Mohanani et al., 2018), and my previous experience in conducting SLR (Garousi et al., 2019;Giray and Tüzün, 2018;Tarhan and Giray, 2017).”
Quote in the paperNot a substantial claim0 of 3 sources checked- cites Kitchenham and Charters, 2007;Wohlin, 2014
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites Durelli et al., 2019;Mohanani et al., 2018
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites Garousi et al., 2019;Giray and Tüzün, 2018;Tarhan and Giray, 2017
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Classified as probably not a substantial claim.
Study design and conductreasoningPapers with a quality score lower than six points out of 12 for empirical studies and three points out of six for non-empirical studies were excluded.
“To maintain a high-quality input of primary studies for this SLR, I decided to exclude the papers with a score lower than six points out of 12 for empirical studies and three points out of six for non-empirical studies.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim.
Study design and conductcitationQuality assessment criteria proposed by Mohanani et al. (2018) were used to assess the quality of primary studies.
“To assess the quality of primary studies, I used the quality assessment criteria (Table 5) proposed by Mohanani et al. (2018).”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Mohanani et al. (2018)
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductnone givenChallenges and solutions were recorded in the spreadsheet without making any changes as written in the primary studies.
“For all primary studies, I recorded the challenges and solutions to the spreadsheet without making any changes as written in the primary studies.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
Study design and conductcitationThe knowledge areas presented in Wan et al. (2019), derived from SWEBOK, were used to classify challenges and solutions.
“I used the knowledge areas presented in Wan et al. (2019), which was derived from SWEBOK (Bourque and Fairley, 2014) to classify the challenges and solutions.”
Quote in the paperSubstantial claim0 of 2 sources checked- cites Wan et al. (2019)
Not checked in the free run.
- cites Bourque and Fairley, 2014
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationTraditional ML algorithms entail a manual feature selection process, which may require domain expertise, statistical analysis of input data, and experiments for building models with different feature sets.
“As briefly explained in Section 2.2, traditional ML algorithms entail a manual feature selection process, which may require domain expertise, statistical analysis of input data, and experiments for building models with different feature sets (Kelleher, 2019).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Kelleher, 2019
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationThe industry is calling for action to resolve the challenges of engineering ML systems and to propose new techniques to cope with the additional complexity of ML systems.
“As these examples show, the industry is calling for action to resolve the challenges of engineering ML systems and to propose new techniques to cope with the additional complexity of ML systems.”
Quote in the paperNot a substantial claim0 of 3 sources checked- cites Lorica and Loukides (2018)
Not checked in the free run.
- cites Heck, 2019
Not checked in the free run.
- cites Sato et al. (2019)
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
Interpretationown dataResearchers show high interest in testing ML systems.
“showing researchers' high interest in testing ML systems”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
Study design and conductcitationOpen coding was conducted to analyze the challenges and solutions.
“I conducted open coding (Miles and Huberman, 1994) to analyze the challenges and solutions.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Miles and Huberman, 1994
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductown dataOne study with a score under the threshold was excluded.
“I excluded one study with a score under the threshold.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationThis change poses two crucial challenges: detecting when concept drift occurs and keeping the models up-to-date.
“This change poses two crucial challenges: detecting when concept drift occurs and keeping the models up-to-date (Maimon and Rokach, 2010).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Maimon and Rokach, 2010
Not checked in the free run.
Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).
PremisecitationThere is a need for a catalog of patterns dedicated to ML-specific problems.
“There is a need for a catalog of patterns dedicated to ML-specific problems (Washizaki et al., 2019b).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Washizaki et al., 2019b
Not checked in the free run.
May combine two claims.
PremisecitationRequirements of ML systems are heavily dependent on available data; different data sets may lead to different needs.
“Requirements of ML systems are heavily dependent on available data; different data sets may lead to different needs (Rahimi et al., 2019;Wan et al., 2019).”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites Rahimi et al., 2019
Not checked in the free run.
- cites Wan et al., 2019
Not checked in the free run.
May combine two claims.
PremisecitationRequirements of ML systems are much more uncertain, and hence they are more difficult to elicit and analyze.
“Requirements are much more uncertain (Belani et al., 2019;Chechik, 2019;Ishikawa and Yoshioka, 2019;Rahimi et al., 2019;Wan et al., 2019), and hence they are more difficult to elicit and analyze.”
Quote in the paperSubstantial claim0 of 3 sources checked- cites Belani et al., 2019
Not checked in the free run.
- cites Chechik, 2019
Not checked in the free run.
- cites Ishikawa and Yoshioka, 2019
Not checked in the free run.
Probably not a substantial claim: may not pass S2 (checkable).
PremisecitationExplainability is quite essential in building trust between an ML system and its users.
“Explainability is quite essential in building trust between an ML system and its users (Nguyen-Duc et al., 2020;Ribeiro et al., 2016).”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites Nguyen-Duc et al., 2020
Not checked in the free run.
- cites Ribeiro et al., 2016
Not checked in the free run.
May combine two claims.
PremisecitationCombining data is not always straightforward since different systems can have other objectives for storing data, hence leading to data semantics heterogeneity and data integration problems.
“In addition, combining data is not always straightforward (Kim et al., 2017;Lwakatare et al., 2019) since different systems can have other objectives for storing data, hence leading to data semantics heterogeneity and data integration problems.”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites Kim et al., 2017 source
Not checked in the free run.
- cites Lwakatare et al., 2019
Not checked in the free run.
May combine two claims.
PremisecitationDistributed architectural patterns are widely used in designing ML systems to cope with high-volume data, which leads to additional complexity in architectural and detailed design.
“Distributed architectural patterns are widely used in designing ML systems to cope with high-volume data (Wan et al., 2019), which leads to additional complexity in architectural and detailed design (Wan et al., 2019).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Wan et al., 2019
Not checked in the free run.
Classified as probably not a substantial claim. May combine two claims.
PremisecitationKnown as 'concept drift,' underlying training data distribution may change, and these changes make the model built on old data inconsistent with the new data.
“Known as”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Tsymbal, 2004
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationTrade-offs among device cost, ML model performance, and data privacy requirements guide the selection of an architectural alternative.
“Trade-offs among device cost, ML model performance, and data privacy requirements guide the selection of an architectural alternative (John et al., 2020b).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites John et al., 2020b
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationDevelopers may need to optimize a tradeoff between the best features and model complexity.
“They may need to optimize a possible tradeoff between the best features and model complexity (Sapp, 2017).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Sapp, 2017
Not checked in the free run.
May combine two claims.
PremisecitationAlthough non-testability is attributed to a lack of reliable test oracle, there are several challenges in testing ML systems.
“Although non-testability is attributed to a lack of reliable test oracle (Weyuker, 1982), there are several challenges in testing ML systems.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Weyuker, 1982 source
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationModel deployment involves some challenges.
“Model deployment involves some challenges (Chen et al., 2020b).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Chen et al., 2020b
Not checked in the free run.
May combine two claims.
PremisecitationAdapting the implementation of a neural network for a different task or transferring a built solution to another domain remains a challenge.
“Adapting the implementation of a neural network for a different task (Zhang et al., 2019a) or transferring a built solution to another domain (Baier et al., 2019) remains a challenge.”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites Zhang et al., 2019a
Not checked in the free run.
- cites Baier et al., 2019
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationAnother essential concern in developing ML models is fairness.
“Another essential concern in developing ML models is fairness (Corbett-Davies and Goel, 2018).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Corbett-Davies and Goel, 2018
Not checked in the free run.
May combine two claims.
InterpretationreasoningModel requirements may not be apparent before starting model development and may be clarified after many experiment iterations.
“Even the model requirements may not be apparent before starting with model development and may be clarified after many experiment iterations.”
Quote in the paperNot a substantial claimMay combine two claims.
PremisecitationDevelopers face challenges in understanding ML algorithms, techniques, and libraries, especially the ones related to DL, and identify a proper model.
“They face challenges in understanding these (Alshangiti et al., 2019;Zhang et al., 2019a;Zhang et al., 2020a), especially the ones related to DL (Alshangiti et al., 2019), and identify a proper model (Islam, 2019).”
Quote in the paperNot a substantial claim0 of 3 sources checked- cites Alshangiti et al., 2019
Not checked in the free run.
- cites Zhang et al., 2019a
Not checked in the free run.
- cites Zhang et al., 2020a
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationDeveloping ML models involves sub-activities such as feature engineering, model training, model evaluation, and model deployment.
“This development endeavor involves some sub-activities, such as feature engineering, model training, model evaluation, and model deployment (Amershi et al., 2019).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Amershi et al., 2019
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationIn 2007, Murphy et al. mentioned the idea of testing ML systems and classified such systems as non-testable.
“In 2007, Murphy et al. (2007) mentioned the idea of testing ML systems and classified such systems as”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Murphy et al., 2007
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationZhou et al. found that OoD-aware ML models are more robust according to their experiments.
“Zhou et al. (2020b) found out that OoD-aware ML models are more robust according to their experiments.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Zhou et al., 2020b
Not checked in the free run.
May combine two claims.
PremisecitationWhile reusing ML libraries is very common, effective, and efficient, reusing ML models in different domains or systems is not straightforward.
“While reusing ML libraries is very common, effective, and efficient, reusing ML models in different domains or systems is not straightforward (Alvarez-Rodríguez et al., 2019;Amershi et al., 2019;Baier et al., 2019;Guo et al., 2019;Sankaran et al., 2017).”
Quote in the paperNot a substantial claim0 of 3 sources checked- cites Alvarez-Rodríguez et al., 2019
Not checked in the free run.
- cites Amershi et al., 2019
Not checked in the free run.
- cites Baier et al., 2019
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationFor DL models, feature engineering is performed implicitly during model training.
“For DL models, feature engineering is performed implicitly during model training (Amershi et al., 2019).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Amershi et al., 2019
Not checked in the free run.
May combine two claims.
PremisecitationDistributed systems require further knowledge and experience and additional cost and management of associated hardware and software.
“Distributed systems require further knowledge and experience and additional cost and management of associated hardware and software (Arpteg et al., 2018).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Arpteg et al., 2018
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationImbalanced datasets will most likely lead to inaccurate prediction models.
“For instance, imbalanced datasets will most likely lead to inaccurate prediction models (He and Garcia, 2009).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites He and Garcia, 2009 source
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationWell-designed test cases can help in finding many types of defects in ML systems.
“Welldesigned test cases can help in finding many types of defects in ML systems (Humbatova et al., 2020).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Humbatova et al., 2020
Not checked in the free run.
Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).
PremisecitationDL models should be evaluated using various important metrics such as robustness and neuron coverage, besides prediction accuracy.
“Nevertheless, DL models should be evaluated using various important metrics such as robustness and neuron coverage, besides the widely-used prediction accuracy metric (Tian et al., 2020).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Tian et al., 2020
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationGalhotra et al. proposed a technique to measure causal discrimination and managed to detect significant ML model discrimination towards gender, marital status, or race as many as 77.2% of the individuals in datasets.
“Galhotra et al. (2017) proposed a technique to measure causal discrimination and managed to detect significant ML model discrimination towards gender, marital status, or race as many as 77.2% of the individuals in datasets.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Galhotra et al., 2017
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationAccording to the experiments conducted by Haq et al., simulator-generated datasets yielded DNN prediction errors that are similar to those obtained via real-world datasets in autonomous driving domain.
“For instance, according to the experiments conducted by Haq et al. (2020), simulator-generated datasets yielded DNN prediction errors that are similar to those obtained via real-world datasets in autonomous driving domain.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Haq et al., 2020
Not checked in the free run.
May combine two claims.
PremisecitationThe discovery of adversarial examples and evaluation of model robustness are fundamental problems in engineering ML systems.
“The discovery of adversarial examples and evaluation of model robustness are fundamental problems in engineering ML systems (Du et al., 2020;Fujii et al., 2020;Sun et al., 2018).”
Quote in the paperNot a substantial claim0 of 3 sources checked- cites Du et al., 2020
Not checked in the free run.
- cites Fujii et al., 2020
Not checked in the free run.
- cites Sun et al., 2018
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationZhou et al. presented DeepReduce, which helped them reduce testing data 7.5% on average and still reliably estimate the performance of DL models.
“Zhou et al. (2020a) presented their approach named DeepReduce, which helped them reduce testing data 7.5% on average and still reliably estimate the performance of DL models.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Zhou et al., 2020a
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationBased on their preliminary results, neuron coverage is not sufficient to reach reliable conclusions about the quality of DeepRL models.
“Based on their preliminary results, neuron coverage is not sufficient to reach reliable conclusions about the quality DeepRL models (Trujillo et al., 2020).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Trujillo et al., 2020
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationYan et al. claimed that DNN coverage criteria do not have monotonic relations with model quality measured by model accuracy in the presence of adversarial examples.
“Yan et al. (2020) claimed that DNN coverage criteria do not have monotonic relations with model quality measured by model accuracy in the presence of adversarial examples.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Yan et al., 2020
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationGarcia et al. classified these bugs into 13 root causes, 20 bug symptoms, and 18 categories of software components these bugs often affect.
“They classified these bugs into 13 root causes, 20 bug symptoms, and 18 categories of software components these bugs often affect (Garcia et al., 2020).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Garcia et al., 2020
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationMutation Testing (MuT) is a fault-based testing technique that claims to address the shortcomings of coverage criteria.
“Mutation Testing (MuT) is a fault-based testing technique that claims to address the shortcomings of coverage criteria (Jia and Harman, 2010;Zhang et al., 2018c).”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites Jia and Harman, 2010
Not checked in the free run.
- cites Zhang et al., 2018c
Not checked in the free run.
May combine two claims.
PremisecitationBased on the experiments conducted by Harel-Canada et al., higher neuron coverage led to fewer defects detected, less natural inputs, and more biased prediction preferences.
“Based on the experiments conducted by Harel-Canada et al. (2020), higher neuron coverage led to fewer defects detected, less natural inputs, and more biased prediction preferences.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Harel-Canada et al., 2020
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationKim et al. and Sekhon and Fleming state that existing coverage criteria are not sufficiently fine-grained to capture subtle behaviors exhibited by ML systems.
“Kim et al. (2019) and Sekhon and Fleming (2019) state that existing coverage criteria are not sufficiently fine-grained to capture subtle behaviors exhibited by ML systems.”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites Kim et al., 2019
Not checked in the free run.
- cites Sekhon and Fleming, 2019
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationLi et al. could not find a strong correlation between the number of misclassified inputs in a test set and its structural coverage.
“Although researchers try to develop new coverage criteria appropriate for DL models, Li et al. (2019) could not find a strong correlation between the number of misclassified inputs in a test set and its structural coverage.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Li et al., 2019
Not checked in the free run.
May combine two claims.
PremisecitationThe size of test case sets is enormous, and missing test cases is a more challenging problem for ML systems.
“The size of test case sets is enormous, and missing test cases is a more challenging problem for ML systems (Khomh et al., 2018).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Khomh et al., 2018 source
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationFlaky tests are observed frequently and may cause problems in software development process.
“Flaky tests are observed frequently and may cause problems in software development process (Luo et al., 2014).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Luo et al., 2014
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationViolation of a metamorphic relation points to a defect.
“Violation of a metamorphic relation points to a defect (Zhou et al., 2015).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Zhou et al., 2015 source
Not checked in the free run.
May combine two claims.
PremisecitationFailing to update the code logic accordingly as ML models evolve may introduce bugs and fail to achieve the optimal performance of ML models, and further negatively influence the entire ML-powered system.
“Failing to update the code logic accordingly as ML models evolve may introduce bugs and fail to achieve the optimal performance of ML models, and further negatively influence the entire ML-powered system (Peng et al., 2020).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Peng et al., 2020
Not checked in the free run.
May combine two claims.
InterpretationreasoningSuch mechanisms can detect performance degradation and take the required actions by re-training ML models with new data.
“Such mechanisms can detect performance degradation and take the required actions by re-training ML models with new data.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationHulten breaks down the ML process into five stages: getting data to model, feature engineering, modeling, deployment, and maintenance.
“For instance, Hulten (2018) breaks down the ML process into five stages: (1) getting data to model, (2) feature engineering, (3) modeling, (4) deployment, and (5) maintenance.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Hulten, 2018
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationShams demonstrates a three-phase ML product development process to shorten the delivery time from idea generation to deployment.
“Shams (2018) demonstrates a threephase ML product development process to shorten the delivery time from idea generation to deployment.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Shams, 2018 source
Not checked in the free run.
May combine two claims.
PremisecitationCanary release approach may limit the possible negative impact of the new ML model and help to cope with the risk of deploying an updated ML model.
“Canary release approach may limit the possible negative impact of the new ML model and help to cope with the risk of deploying an updated ML model (Yokoyama, 2019).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Yokoyama, 2019
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationExperiment components can include hardware, platform, source code, configuration, training data, and model state.
“These components can include hardware (GPU models primarily), platform (operating system and installed packages), source code (model training and pre-processing), configuration (model configuration and pre-processing settings), training data (input signals and target values), and model state (versions of trained models) (Arpteg et al., 2018;Wan et al., 2019;Yokoyama, 2019).”
Quote in the paperNot a substantial claim0 of 3 sources checked- cites Arpteg et al., 2018
Not checked in the free run.
- cites Wan et al., 2019
Not checked in the free run.
- cites Yokoyama, 2019
Not checked in the free run.
- Rests on an assumption: Each experiment may have many components that affect the outcome, i.e., the ML model.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationIslam et al. found that developers are mostly fixing data dimensions and neural network connectivity.
“They found that developers are mostly fixing data dimensions and neural network connectivity (Islam et al., 2020).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Islam et al., 2020
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationRao shows the typical steps involved in the model development lifecycle: define problem, collect/cleanse/prepare data, build/train model, tune hyperparameters/validate model, deploy to production.
“Rao (2019) shows the typical steps involved in the model development lifecycle: (1) define the problem, (2) collect, cleanse, and prepare data, (3) build and train model, (4) tune hyperparameters and validate model, and (5) deploy to production.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Rao, 2019
Not checked in the free run.
May combine two claims.
PremisecitationExperiment management is a recent and complicated challenge in engineering ML systems.
“Therefore, experiment management is a recent and complicated challenge in engineering ML systems (Hazelwood et al., 2018;Zaharia et al., 2018).”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites Hazelwood et al., 2018
Not checked in the free run.
- cites Zaharia et al., 2018
Not checked in the free run.
- Rests on an assumption: Some large-scale ML systems involve configuration management at a higher level, i.e., keeping track of experiments.
- Rests on an assumption: Each experiment may have many components that affect the outcome, i.e., the ML model.
- Rests on an assumption: It is possible to use automated metaoptimization methods and conduct lots of experiments without human intervention.
May combine two claims.
PremisecitationWhile versioning data is a vital need, the methods and tools to support this are not mature yet.
“While versioning data is a vital need (Alvarez-Rodríguez et al., 2019;Amershi et al., 2019;Belani et al., 2019;Wan et al., 2019;Yokoyama, 2019), the methods and tools to support this is not mature yet (Amershi et al., 2019).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Amershi et al., 2019
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationPeng et al. observed that as ML models evolve, it poses maintenance challenges on the code logic that should coevolve.
“Peng et al. (2020) observed that as ML models evolve, it poses maintenance challenges on the code logic that should coevolve.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Peng et al., 2020
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationKubernetes ecosystem may ease maintenance and problem localization.
“For instance, Kubernetes ecosystem may ease maintenance and problem localization (Benton, 2020).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Benton, 2020 source
Not checked in the free run.
May combine two claims.
PremisecitationWhile developers fix bugs, they may introduce adversarial vulnerabilities and new bugs.
“While developers fix bugs, they may introduce adversarial vulnerabilities and new bugs (Islam et al., 2020).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Islam et al., 2020
Not checked in the free run.
The quote only partly states this claim.
Interpretationnone givenImproper experiment management may result in experiments and their results not being analyzed, reported, and shared appropriately.
“(3) experiments and their results may not be analyzed, reported, and shared appropriately.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationExperimentation activities generate many artifacts (datasets, hyperparameters, models, libraries, etc.), which should be versioned.
“Whether done automatically or manually, experimentation activities generate many artifacts (datasets, hyperparameters, models, libraries, etc.), which should be versioned (Golovin et al., 2017).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Golovin et al., 2017
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationIn risky domains, automated checks can be used to trigger notifications if predefined thresholds are violated.
“In risky domains, automated checks can be used to trigger notifications if predefined thresholds are violated (Baier et al., 2019).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Baier et al., 2019
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationVersioning data involves keeping track of datasets, metadata of datasets, and their relationships to ML models.
“Versioning data does not only involve keeping track of datasets but also keeping track of metadata of datasets (Gebru et al., 2018) and their relationships to ML models (Amershi et al., 2019).”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites Gebru et al., 2018
Not checked in the free run.
- cites Amershi et al., 2019
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationThe steps are not executed as a waterfall process; there is feedback from deployment to problem definition and an iteration from building a model to deployment.
“The steps are not executed as a waterfall process; there is feedback from deployment to problem definition and an iteration from building a model to deployment (Rao, 2019).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Rao, 2019
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationPan observed that bug fixes may cause both an increase and decrease in robustness of DL models.
“In another study, Pan (2020) observed that bug fixes may cause both an increase and decrease in robustness of DL models.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Pan, 2020
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationAnother option is to augment training data with the guidance of newly identified input data and retrain ML model.
“Another option is to augment training data with the guidance of newly identified input data and retrain ML model (Ren et al., 2020).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Ren et al., 2020
Not checked in the free run.
May combine two claims.
PremisecitationWith the emergence of many general-purpose ML libraries, services, platforms combined with the accumulation of ML experience, developing ML systems has become relatively easy and cheap, whereas maintaining them over time is difficult and expensive.
“With the emergence of many general-purpose ML libraries, services, platforms combined with the accumulation of ML experience, developing ML systems has become relatively easy and cheap, whereas maintaining them over time is difficult and expensive (Sculley et al., 2015).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Sculley et al., 2015
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationNewly identified data can be used directly for adaptive retraining.
“Newly identified data can be used directly for adaptive retraining (Stocco and Tonella, 2020).”
- Also stated in: § 6.1 POTENTIAL RESEARCH DIRECTIONS FROM THE SE PERSPECTIVE · plain
“The identified data can be used for adaptive retraining.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Stocco and Tonella, 2020
Not checked in the free run.
- Also stated in: § 6.1 POTENTIAL RESEARCH DIRECTIONS FROM THE SE PERSPECTIVE · plain
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationAn ML engineer needs to understand the mathematical and statistical background of models for proper model fitting and tuning.
“An ML engineer need to understand the mathematical and statistical background of models for proper model fitting and tuning (Alshangiti et al., 2019).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Alshangiti et al., 2019
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationMoreb et al. proposed a process for engineering ML systems in health informatics domain.
“Moreb et al. (2020) proposed a process for engineering ML systems in health informatics domain.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Moreb et al., 2020 source
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationA software engineer may have to deal with distributed systems, web services, and multi-threaded programming.
“A software engineer may have to deal with distributed systems, web services, and multi-threaded programming (Alshangiti et al., 2019).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Alshangiti et al., 2019
Not checked in the free run.
May combine two claims.
PremisecitationDue to the uncertainties in the ML process, it is hard to estimate effort and do planning.
“Due to the uncertainties in the ML process, it is hard to estimate effort and do planning (Wan et al., 2019).”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Wan et al., 2019
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationThe popularity of ML has brought many tools, mostly open-source.
“The popularity of ML has brought many tools, mostly open-source (Louridas and Ebert, 2016).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Louridas and Ebert, 2016
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationAccording to Sensemaking-Coevolution-Implementation Theory, sensemaking is one of the core activities in engineering software systems.
“According to Sensemaking-Coevolution-Implementation Theory, sensemaking is one of the core activities in engineering software systems (Ralph, 2015).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Ralph, 2015 source
Not checked in the free run.
May combine two claims.
PremisecitationWhile data scientists may be pragmatic when coding as long as they achieve the desired results, software engineers may care about code quality and maintainability.
“For example, while data scientists may be pragmatic when coding as long as they achieve the desired results, software engineers may care about code quality and maintainability (Arpteg et al., 2018).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Arpteg et al., 2018
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationComponents with ML capabilities are becoming an architectural part of software systems, sharing cross-cutting functional and non-functional concerns.
“Components with ML capabilities are becoming an architectural part of software systems, sharing cross-cutting functional and non-functional concerns (Nguyen-Duc et al., 2020).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Nguyen-Duc et al., 2020
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationPeople having different backgrounds have cultural and language differences.
“People having different backgrounds have cultural (Arpteg et al., 2018) and language differences.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Arpteg et al., 2018
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationEngineering complex systems, such as ML systems, is a complicated endeavor with organizational and technical aspects.
“Engineering complex systems, such as ML systems, is a complicated endeavor with organizational and technical aspects (Clarke et al., 2016).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Clarke et al., 2016
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationWolf and Paine emphasizes the difficulty of sensemaking for business context, ML algorithms, libraries, and models in ML systems engineering projects.
“Wolf and Paine (2020) emphasizes the difficulty of sensemaking for business context, ML algorithms, libraries, and models in ML systems engineering projects.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Wolf and Paine, 2020
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationAlvarez-Rodríguez et al. identified the formation of a workforce equipped with the required skills to tackle different types of problems in various domains as a challenge.
“Alvarez-Rodríguez et al. (2019) identified the formation of a workforce equipped with the required skills to tackle different types of problems in various domains as a challenge.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Alvarez-Rodríguez et al., 2019
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationA data preparer requires good data slicing and dicing skills for proper data cleaning and feature preprocessing.
“A data preparer requires a good data slicing and dicing skills for proper data cleaning and feature preprocessing (Alshangiti et al., 2019).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Alshangiti et al., 2019
Not checked in the free run.
May combine two claims.
PremisecitationMonitoring for potential performance degradation on production and high-volume data processing are two important design considerations for ML systems.
“Monitoring for potential performance degradation on production (Wan et al., 2019;Yokoyama, 2019) and high-volume data processing (de Souza Nascimento et al., 2019;Wan et al., 2019) are two important design considerations for ML systems.”
Quote in the paperNot a substantial claim0 of 3 sources checked- cites Wan et al., 2019
Not checked in the free run.
- cites Yokoyama, 2019
Not checked in the free run.
- cites de Souza Nascimento et al., 2019
Not checked in the free run.
Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).
InterpretationreasoningThe reason for having much fewer papers in journals may be due to the fact that this topic is still in its infancy.
“The reason for having much fewer papers in journals may be due to the fact that this topic is still in its infancy.”
Quote in the paperNot a substantial claimMay combine two claims.
PremisecitationThere are some early attempts to determine the ML process' maturity and how ML components are used in larger software systems.
“There are some early attempts to determine the ML process' maturity (Amershi et al., 2019) and how ML components are used in larger software systems (Lwakatare et al., 2019).”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites Amershi et al., 2019
Not checked in the free run.
- cites Lwakatare et al., 2019
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationThere are some efforts to bring various tools together to support the effective development of ML systems.
“There are some efforts to bring various tools together to support the effective development of ML systems (Smith et al., 2020).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Smith et al., 2020
Not checked in the free run.
May combine two claims.
PremisecitationThis diverse set of tools have emerged with challenges, such as compatibility, integration of various components, required knowledge and experience.
“However, this diverse set of tools have emerged with challenges, such as compatibility (Rahman et al., 2019), integration of various components (Rahman et al., 2019), required knowledge and experience, etc.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Rahman et al., 2019
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationML systems' development involves a diverse set of frameworks, tools, libraries, and programming languages.
“ML systems' development involves a diverse set of frameworks, tools, libraries, and programming languages (Rahman et al., 2019).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Rahman et al., 2019
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationWhile there are generic roles such as data scientist, ML engineer, data analyst, ML software engineer, ML researcher, and software engineer, there are more specific roles, such as data evangelist, data preparer, data shaper, data analyzer.
“While there are generic roles such as data scientist, ML engineer, data analyst, ML software engineer, ML researcher, and software engineer, there are more specific roles, such as data evangelist, data preparer, data shaper, data analyzer (Kim et al., 2017).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Kim et al., 2017 source
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductreasoningBroad terms were used to form search keywords to reduce the risk of excluding potentially relevant studies.
“I used broad terms to form search keywords to reduce the risk of excluding potentially relevant studies.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
Study design and conductnone givenThe study covers primary studies published until 31 December 2020.
“Date: This study covers primary studies published until the end of 2020, i.e., 31 December 2020.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationThe success of these coverage criteria is questionable.
“However, the success of these coverage criteria is questionable (Li et al., 2019).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Li et al., 2019
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationDrift detectors may be developed to identify data deviating from training data distribution.
“To maintain the performance of an ML system above a level that meets customer expectations, drift detectors may be developed to identify data deviating from training data distribution (Stocco and Tonella, 2020).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Stocco and Tonella, 2020
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationA set of guidelines was published to ensure ML systems' quality.
“A set of guidelines was published to ensure ML systems' quality (QA4AI, 2020).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites QA4AI, 2020
Not checked in the free run.
Classified as probably not a substantial claim. May combine two claims.
Study design and conductreasoningThe primary studies have been selected from SE venues to reflect an SE perspective on ML systems engineering, and papers published in AI, ML, DL, Data Science, Data Management venues have been excluded.
“Perspective: The primary studies have been selected from SE venues to reflect an SE perspective on ML systems engineering.Therefore, papers published in AI, ML, DL, Data Science, Data Management venues have been excluded.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductreasoningWith this approach, precision was decreased and recall increased, finding more candidate papers to be assessed for specifying the final set of primary studies.
“With this approach, I decreased precision and increased recall found more candidate papers to be assessed for specifying the final set of primary studies.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationThe course discusses how to take an idea and an ML model developed by data scientists and deploy it as part of a scalable and maintainable system.
“It discusses how to take an idea and an ML model developed by data scientists and deploy it as part of a scalable and maintainable system.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Kästner and Kang, 2020
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationRudraraju and Boyanapally focused on developing a data quality model for ML systems.
“Rudraraju and Boyanapally, (2019) focused on developing a data quality model for ML systems.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Rudraraju and Boyanapally, 2019
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductnone givenPersonal bias might have been introduced during the application of inclusion and exclusion criteria.
“Personal bias might have been introduced during the application of inclusion and exclusion criteria.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationThey identified new data quality attributes that are not present in the Data Quality model defined in the standard ISO/IEC 25012.
“They identified new data quality attributes that are not present in the Data Quality model defined in the standard ISO/IEC 25012.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Rudraraju and Boyanapally, 2019
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationThe QA4AI Consortium was established in 2018 in Japan to discuss the quality assurance of ML systems.
“As an example of the fulfillment of this need, the QA4AI Consortium was established in 2018 in Japan to discuss the quality assurance of ML systems (Fujii et al., 2020).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Fujii et al., 2020
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationSome concerns, such as some non-functional requirements, may be domain-specific.
“Some of the concerns, such as some non-functional requirements, may be domain-specific (Nguyen-Duc et al., 2020) and can be discovered by exploring various ML problems.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Nguyen-Duc et al., 2020
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationThe 2020 Future of Jobs Survey reveals that the top increasingly strategic job roles are data analysts and scientists, AI and ML specialists, robotics engineers, software and application developers, and digital transformation specialists.
“The 2020 version of the Future of Jobs Survey reveals that the top increasingly strategic job roles are data analysts and scientists, AI and ML specialists, robotics engineers, software and application developers, and digital transformation specialists (World Economic Forum, 2020).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites World Economic Forum, 2020
Not checked in the free run.
Classified as probably not a substantial claim. May combine two claims.
Study design and conductnone givenThe study comprises studies published in peer-reviewed academic venues, and grey literature has been excluded.
“Type of Literature: This study comprises studies published in peer-reviewed academic venues.Grey literature, e.g., papers only published in arxiv.org,blogs, videos, etc., have been excluded.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductreasoningBackward and forward snowballing along with a manual search were carried out to mitigate the risk of missing some relevant studies.
“To mitigate the risk of missing some relevant studies, I also carried out both backward and forward snowballing along with a manual search.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationTools to estimate computation and memory requirements, such as DNNMem, may help developers.
“Tools to estimate computation and memory requirements, such as DNNMem (Gao et al., 2020b), may help developers.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Gao et al., 2020b
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductnone givenFive widely used online databases in SMS and SLR studies in computer science and software engineering were searched.
“I also searched for five widely used online databases in SMS and SLR studies in computer science and software engineering.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationResearchers analyzed and classified these bugs.
“Researchers analyzed and classified these bugs (Garcia et al., 2020).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Garcia et al., 2020
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationOne of the pioneer courses is the Software Engineering for AI-Enabled Systems course offered at Carnegie Mellon University.
“One of the pioneer courses is the”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Kästner and Kang, 2020
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationAutomating tests has become a necessity to cope with large-scale complex systems.
“Automating tests has become a necessity to cope with large-scale complex systems (Mosley and Posey, 2002).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Mosley and Posey, 2002
Not checked in the free run.
May combine two claims.
PremisecitationThere are many processes for ML lifecycle and traditional software development.
“While there are many processes for ML lifecycle (Hulten, 2018;Rao, 2019;Shams, 2018) and traditional software development (Klünder et al., 2019), a set of harmonized practices for developing ML systems is required (Alvarez-Rodríguez et al., 2019;Belani et al., 2019;de Souza Nascimento et al., 2019).”
Quote in the paperNot a substantial claim0 of 3 sources checked- cites Hulten, 2018
Not checked in the free run.
- cites Rao, 2019
Not checked in the free run.
- cites Shams, 2018 source
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationWan et al. only mentioned the difficulty of effort estimation in ML projects.
“Wan et al.only mentioned the difficulty of effort estimation in ML projects (Wan et al., 2019).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Wan et al., 2019
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationThere are some examples of using ML algorithms for project management.
“While there are some examples of using ML algorithms for project management (two examples in ML for SE by Nassif et al. (2016) and Pospieszny et al. (2018)), our primary study pool does not include any practices for managing ML systems development projects.”
Quote in the paperNot a substantial claim0 of 2 sources checkedProbably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationValidity considerations are applicable for SMS and SLR studies, like empirical studies.
“Validity considerations are applicable for SMS and SLR studies, like empirical studies (Petersen et al., 2008;Petersen et al., 2015).”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites Petersen et al., 2008
Not checked in the free run.
- cites Petersen et al., 2015 source
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
InterpretationreasoningThere may be other relevant studies published in AI, ML, DL, Data Science, and Data Management venues.
“There may be other relevant studies published in AI, ML, DL, Data Science, and Data Management venues.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationOne of the challenges to be addressed is the identification of metamorphic relations that should be satisfied by an ML system.
“One of the challenges to be addressed is the identification of metamorphic relations that should be satisfied by an ML system (Kanewala and Bieman, 2014).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Kanewala and Bieman, 2014 source
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationThe course takes a SE perspective on building software systems with a significant AI/ML component.
“The course takes a SE perspective on building software systems with a significant AI/ML component.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Kästner and Kang, 2020
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductnone givenGoogle Scholar was also searched to enrich the paper pool.
“Besides these five databases, I also searched Google Scholar to enrich the paper pool.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).
InterpretationreasoningMore testing can be conducted via test automation to reduce the risks of ML systems.
“More testing can be conducted via test automation to reduce the risks of ML systems.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationTwo promising techniques for designing test cases are metamorphic testing techniques and leveraging adversarial techniques to detect erroneous behavior.
“Two promising techniques for designing test cases are metamorphic testing techniques and leveraging adversarial techniques to detect erroneous behavior (Mjeda and Botterweck, 2019) on which further research is needed.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Mjeda and Botterweck, 2019
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductnone givenThe threats to this SLR's validity are mainly related to the specification of the candidate pool of papers, primary study selection bias, data extraction, and data synthesis.
“The threats to this SLR's validity are mainly related to the specification of the candidate pool of papers, primary study selection bias, data extraction, and data synthesis.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductreasoningTo minimize bias and errors, when inclusion/exclusion could not be decided by reading the abstract, the full text was scanned for the final decision.
“To minimize this type of bias and errors whenever I could not decide on inclusion/exclusion by reading the abstract, I scanned the full text for the final decision.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
Premisenone givenThe validity of data extraction directly affects this study's results.
“The validity of data extraction is another essential aspect that directly affects this study's results.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductnone givenOpen coding was applied iteratively and incrementally to identify the challenges and solutions (RQ3).
“I applied the open coding technique iteratively and incrementally to identify the challenges and solutions (RQ3).”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductreasoningThis coding process potentially entails some researcher bias.
“This coding process potentially entails some researcher bias.”
Quote in the paperNot a substantial claimMay combine two claims.
Study design and conductcitationIn general, primary study selection, data extraction, and synthesis are subject to researcher bias, and this bias may be higher than those SLRs with multiple authors.
“In general, primary study selection, data extraction, and synthesis are subject to researcher bias, and this bias may be higher than those SLRs with multiple authors.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
Study design and conductreasoningExisting categories in the literature were used where possible (RQ1, RQ2, and SE knowledge areas in RQ3) to ensure the accuracy of the extracted data.
“In order to ensure the accuracy of the extracted data, I used existing categories in the literature where possible (RQ1, RQ2, and SE knowledge areas in RQ3).”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductreasoningRelevant data in primary studies were mapped to specified groups to decrease the risk of researcher bias.
“I aimed to decrease the risk of researcher bias by mapping the relevant data in primary studies to the specified groups.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationSLRs with single authors can be found in SE and other research areas.
“Despite its disadvantages, we can find SLRs with single authors in SE (Malhotra, 2015) and other research areas (Mitchell, 2011).”
Quote in the paperNot a substantial claim0 of 2 sources checked
The authors state this as an assumption.
The search space can be restricted by assuming a search saturation effect, such that checking the first 200 results of each query in each database is sufficient.
“Therefore, as others have done (Adams et al., 2016;Garousi and Mäntylä, 2016;Garousi et al., 2018;Godin et al., 2015), I had to restrict the search space by assuming a”
The paper relies on this without support.
The primary studies' abstracts are representative of the full papers for the purpose of term frequency analysis.
“I created a bar chart of the most frequently used 50 terms in the primary studies' abstracts (Figure 5).”
The paper relies on this without support.
The classification of SE challenges based on the main SE knowledge areas presented in Wan et al. (2019) is suitable for organizing the identified challenges.
“The identified SE challenges are classified based on the main SE knowledge areas presented in (Wan et al., 2019).”
The paper relies on this without support.
The outputs of an ML model may be used by many other components implicitly.
“The outputs of an ML model may be used by many other components implicitly.”
The paper relies on this without support.
ML system performance is subject to change due to its non-deterministic nature.
“Moreover, ML systems should be monitored continuously since their performance is subject to change due to their non-deterministic nature.”
The paper relies on this without support.
ML models are dependent on ML frameworks.
“ML models are also dependent on ML frameworks.”
The paper relies on this without support.
Some large-scale ML systems involve configuration management at a higher level, i.e., keeping track of experiments.
“While it is necessary to keep track of changes in data and ML models, some largescale ML systems involve configuration management at a higher level, i.e., keeping track of experiments.”
The paper relies on this without support.
Each experiment may have many components that affect the outcome, i.e., the ML model.
“Each experiment may have many components that affect the outcome, i.e., the ML model.”
The paper relies on this without support.
It is possible to use automated metaoptimization methods and conduct lots of experiments without human intervention.
“Also, it is possible to use automated metaoptimization methods and conduct lots of experiments without human intervention.”
The paper relies on this without support.
One of the hardest parts of maintaining ML systems is to keep the performance (such as accuracy) at a certain level or improve it if needed.
“One of the hardest parts of maintaining ML systems is to keep the performance (such as accuracy) at a certain level or improve it if needed.”
The paper relies on this without support.
There are some process models, phases, activities that document steps to develop and maintain ML models.
“There are some process models, phases, activities that document steps to develop and maintain ML models.”
The paper relies on this without support.
Adding ML components to a software system involves additional actions to be performed.
“Adding ML components to a software system involves additional actions to be performed.”
The paper relies on this without support.
Having the required people with proper skill sets is necessary but not sufficient to engineer ML systems.
“Having the required people with proper skill sets is necessary but not sufficient to engineer ML systems.”
The paper relies on this without support.
Specifying ML components' requirements requires a different perspective than traditional software components.
“Specifying ML components' requirements requires a different perspective than traditional software components.”
The paper relies on this without support.
ML systems include traditional software components as well.
“Besides, ML systems include traditional software components as well.”
The paper relies on this without support.
Developers deal with additional complications while developing ML systems.
“Developers deal with additional complications while developing ML systems.”
The paper relies on this without support.
Another challenge is the estimation of resource requirements for production environment.
“Another challenge is the estimation of resource requirements for production environment.”
The paper relies on this without support.
Data, ML models, and experiments have become essential configuration items in maintaining ML systems.
“Data, ML models, and experiments have become essential configuration items in maintaining ML systems.”
The paper relies on this without support.
Keeping versions of data and ML models require tools integrated with the development environment.
“Keeping versions of data and ML models require tools integrated with the development environment.”
The paper relies on this without support.
Engineering ML systems is a highly practical area.
“although engineering ML systems is a highly practical area”
The paper relies on this without support.
ML systems engineering brings together two challenging research areas, i.e., SE and ML, and requires various skillsets.
“ML systems engineering brings together two challenging research areas, i.e., SE and ML, and requires various skillsets.”
The paper relies on this without support.
SE venues can provide a representative set to identify state of the art and challenges of engineering ML systems.
“However, I believe that SE venues can provide a representative set to identify state of the art and challenges of engineering ML systems.”
The paper relies on this without support.
More and more software systems are involving ML capabilities.
“More and more software systems are involving ML capabilities.”
The paper relies on this without support.
Advances in machine learning lead to a transition from conventional software development, where algorithms are hard-coded by humans, to ML systems that are materialized through learning from data.
“Advances in machine learning lead to a transition from the conventional view of software development, where algorithms are hard-coded by humans, to ML systems that are materialized through learning from data.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The quality of included studies was assessed using criteria adapted from [18] and suggested by [17].
“The quality of included studies was assessed using the criteria found in Table 2, adapted from criteria used by [18] and suggested by [17].”
From Operationalizing machine learning models: a systematic literature review (2022) · cites this paper as [18]
ML models have an additional data dependency, in that their behavior is learned from data, and they have been characterized as non-deterministic.
“ML models have an additional data dependency, in the sense that their behavior is learned from data, and they have even been characterized as non-deterministic [18,36].”
From Operationalizing machine learning models: a systematic literature review (2022) · cites this paper as [18]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Online testing is said both to identify and miss the same violations
MinorThe sentence says that safety violations identified through online testing could not be identified through online testing. The surrounding comparison concerns offline versus online testing.
“many safety violations identified through online testing”
“could not be identified through online testing (Haq et al., 2020).”
Review method is labeled both SMS and SLR
MinorTable 1 classifies this study as a systematic mapping study (SMS), while the abstract says it was a systematic literature review (SLR).
“This study 2007 -Dec 2020 identify, analyze, summarize, and synthesize the current state of SE research for engineering ML systems SMS using DB search, snowballing, and manual search 141 papers”
“I performed a systematic literature review (SLR).”
Number of studies using multiple methods is misstated
MinorThe text names at least four studies using multiple research methods before claiming that only three such studies account for the total in Figure 6.
“Due to these three primary studies using multiple research methods, the total number of primary studies equals 147 in Figure 6.”
“Two studies (Amershi et al., 2019;Wan et al., 2019) combined interview and survey methods.Two other studies (Humbatova et al., 2020;Zhang et al., 2020a) used thematic analysis and interview methods together.”
Test management is described as unaddressed despite a reported case study
MinorThe research-directions section says researchers have not addressed test management, but the results section reports a case study that used tools and spreadsheets to manage testing.
“Planning, controlling, and monitoring of testing activities for ML systems have not been addressed by researchers.”
“In a case study, Rivero et al. (2020) utilized test management tools and spreadsheets to manage test process in which various teams were involved.”
Cites
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805 · see citing works
Qualitative Data Analysis: An Expanded Sourcebook
(1994) · Journal of Environmental Psychology · doi:10.1016/s0272-4944(05)80231-2 · Cited by 48,420 · see citing works
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 48,293 · see citing works
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380 · see citing works
Rethinking the Inception Architecture for Computer Vision
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., & Wojna, Z. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.308 · Cited by 32,138 · see citing works
Learning Multiple Layers of Features from Tiny Images
Krizhevsky, A. (2024) · doi:10.57702/zp44cu3g · Cited by 23,890 · see citing works
Pan, S. J., & Yang, Q. (2010) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2009.191 · Cited by 23,387 · see citing works
Artificial intelligence: a modern approach
Kumar, D. A., Balasubramanian, S., Byeon, D. H., & Manerkar, P. G. V. (1995) · Choice Reviews Online · doi:10.5860/choice.33-1577 · Cited by 22,182 · see citing works
(1994) · Cited by 21,711 · see citing works
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555 · see citing works
(2016) · Cited by 11,679 · see citing works
He, H., & Garcia, E. A. (2009) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2008.239 · Cited by 10,359 · see citing works
Machine learning: Trends, perspectives, and prospects
Jordan, M. I., & Mitchell, T. M. (2015) · Science · doi:10.1126/science.aaa8415 · Cited by 9,955 · see citing works
Learning Transferable Architectures for Scalable Image Recognition
Zoph, B., Vasudevan, V., Shlens, J., & Le, Q. V. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00907 · Cited by 6,631 · see citing works
Weiss, K., Khoshgoftaar, T. M., & Wang, D. (2016) · Journal of Big Data · doi:10.1186/s40537-016-0043-6 · Cited by 6,260 · see citing works
Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
Xiao, H., Rasul, K., & Vollgraf, R. (2017) · arXiv · doi:10.48550/arxiv.1708.07747 · Cited by 6,108 · see citing works
Artificial intelligence in healthcare: past, present and future
Jiang, F., Jiang, Y., Zhi, H., Dong, Y., Li, H., Ma, S., Wang, Y., Dong, Q., Shen, H., & Wang, Y. (2017) · Stroke and Vascular Neurology · doi:10.1136/svn-2017-000101 · Cited by 4,837 · see citing works
Reading digits in natural images with unsupervised feature learning
Netzer, Y. (2024) · doi:10.57702/e7wwo0zk · Cited by 4,261 · see citing works
Wohlin, C. (2014) · Proceedings of the 18th International Conference on Evaluation and Assessment in Software Engineering · doi:10.1145/2601248.2601268 · Cited by 4,234 · see citing works
Systematic Mapping Studies in Software Engineering
Petersen, K., Feldt, R., Mujtaba, S., & Mattsson, M. (2008) · Electronic Workshops in Computing · doi:10.14236/ewic/ease2008.8 · Cited by 3,239 · see citing works
Recent Trends in Deep Learning Based Natural Language ProcessingPreprint
Young, T., Hazarika, D., Poria, S., & Cambria, E. (2017) · IEEE Computational Intelligence Magazine · doi:10.48550/arxiv.1708.02709 · Cited by 3,133 · see citing works
Recent Trends in Deep Learning Based Natural Language Processing [Review Article]
Young, T., Hazarika, D., Poria, S., & Cambria, E. (2018) · IEEE Computational Intelligence Magazine · doi:10.1109/mci.2018.2840738 · Cited by 2,921 · see citing works
Guidelines for conducting systematic mapping studies in software engineering: An update
Petersen, K., Vakkalanka, S., & Kuzniarz, L. (2015) · Information and Software Technology · doi:10.1016/j.infsof.2015.03.007 · Cited by 2,669 · see citing works
Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J. W., Wallach, H., Daumé, H., & Crawford, K. (2018) · arXiv · doi:10.48550/arxiv.1803.09010 · Cited by 2,474
Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J. W., Wallach, H., III, H. D., & Crawford, K. (2021) · Communications of the ACM · doi:10.1145/3458723 · Cited by 2,469 · see citing works
Data Mining and Knowledge Discovery Handbook
(2010) · doi:10.1007/978-0-387-09823-4 · Cited by 2,190 · see citing works
NewsWeeder: Learning to Filter Netnews
Lang, K. (1995) · Elsevier eBooks · doi:10.1016/b978-1-55860-377-6.50048-7 · Cited by 2,119 · see citing works
DeepDriving: Learning Affordance for Direct Perception in Autonomous Driving
Chen, C., Seff, A., Kornhauser, A., & Xiao, J. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.312 · Cited by 1,876 · see citing works
Patterns of Enterprise Application Architecture
Fowler, M. (2002) · Cited by 1,809 · see citing works
An Analysis and Survey of the Development of Mutation Testing
Jia, Y., & Harman, M. (2010) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2010.62 · Cited by 1,752 · see citing works
Requirements Engineering: Processes and Techniques
Kotonya, G., & Sommerville, I. (1998) · Lancaster EPrints (Lancaster University) · Cited by 1,702 · see citing works
(2018) · International Conference on Learning Representations · Cited by 1,621 · see citing works
Agile Software Development, Principles, Patterns, and Practices
Martin, R. (2002) · Cited by 1,466 · see citing works
Scaling up the accuracy of Naive-Bayes classifiers: a decision-tree hybrid
Kohavi, R. (1996) · Cited by 1,404 · see citing works
Tian, Y., Pei, K., Jana, S., & Ray, B. (2018) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3180155.3180220 · Cited by 1,400 · see citing works
Machine learning in manufacturing: Advantages, challenges, and applications
Wuest, T., Weimer, D., Irgens, C., & Thoben, K.-D. (2016) · Production & Manufacturing Research · doi:10.1080/21693277.2016.1192517 · Cited by 1,319 · see citing works
Design and Use of Software Architectures: Adopting and Evolving a Product-Line Approach
Bosch, J. (2000) · Medical Entomology and Zoology · Cited by 1,167 · see citing works
The oracle problem in software testing: A survey
Barr, E. T., Harman, M., McMinn, P., Shahbaz, M., & Yoo, S. (2015) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2014.2372785 · Cited by 1,104 · see citing works
Fastai: A Layered API for Deep Learning
Howard, J., & Gugger, S. (2020) · Information · doi:10.3390/info11020108 · Cited by 992 · see citing works
Automated software test data generation
Korel, B. (1990) · IEEE Transactions on Software Engineering · doi:10.1109/32.57624 · Cited by 973 · see citing works
Godin, K., Stapleton, J., Kirkpatrick, S. I., Hanning, R. M., & Leatherdale, S. T. (2015) · Systematic Reviews · doi:10.1186/s13643-015-0125-0 · Cited by 959 · see citing works
The problem of concept drift: definitions and related work
Tsymbal, A. (2004) · Cited by 936 · see citing works
(2015) · neural information processing systems · Cited by 912 · see citing works
Machine Learning Testing: Survey, Landscapes and Horizons
Zhang, J. M., Harman, M., Ma, L., & Liu, Y. (2022) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2019.2962027 · Cited by 897 · see citing works
A data-driven approach to predict the success of bank telemarketing
Moro, S., Cortez, P. C., & Rita, P. (2014) · Decision Support Systems · doi:10.1016/j.dss.2014.03.001 · Cited by 885 · see citing works
McMahan, H. B., Holt, G., Sculley, D., Young, M., Ebner, D., Grady, J., Nie, L., Phillips, T., Davydov, E. V., Golovin, D., et al. (2013) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/2487575.2488200 · Cited by 870 · see citing works
The master algorithm: how the quest for the ultimate learning machine will remake our world
(2016) · Choice Reviews Online · doi:10.5860/choice.194685 · Cited by 808 · see citing works
Deep Learning for Finance: Deep Portfolios
Heaton, J. B., Polson, N., & Witte, J. H. (2016) · Applied Stochastic Models in Business and Industry · doi:10.1002/asmb.2209 · Cited by 766 · see citing works
Evidence-Based Software Engineering and Systematic Reviews
Kitchenham, B. A., Budgen, D., & Brereton, P. (2015) · Chapman & Hall/CRC eBooks · doi:10.1201/b19467 · Cited by 766 · see citing works
Three naive Bayes approaches for discrimination-free classification.
Calders, T., & Verwer, S. (2010) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-010-0190-x · Cited by 764 · see citing works
Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining
Ghani, R., USA, K. 1. T. 1. A. S. I. C. O. K. D. A. D. M. 2. C. I., Association for Computing Machinery-Digital Library, ACM Special Interest Group on Knowledge Discovery in Data, ACM Special Interest Group on Management of Data, & SIGs, A. (2013) · doi:10.1145/2487575 · Cited by 753 · see citing works
Collaborative filtering and deep learning based recommendation system for cold start items
Jian, W., He, J., Chen, K., Zhou, Y., & Tang, Z. (2016) · Expert Systems with Applications · doi:10.1016/j.eswa.2016.09.040 · Cited by 713 · see citing works
DeepGauge: multi-granularity testing criteria for deep learning systems
Ma, L., Juefei-Xu, F., Zhang, F., Sun, J., Xue, M., Li, B., Chen, C., Su, T., Li, L., Liu, Y., et al. (2018) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3238147.3238202 · Cited by 710 · see citing works
Discrimination-aware data mining
Pedreshi, D., Ruggieri, S., & Turini, F. (2008) · Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1401890.1401959 · Cited by 676 · see citing works
Larman, C. (2004) · Prentice Hall PTR eBooks · Cited by 628 · see citing works
A systematic review of machine learning techniques for software fault prediction
Malhotra, R. (2015) · Applied Soft Computing · doi:10.1016/j.asoc.2014.11.023 · Cited by 620 · see citing works
On testing non-testable programs
Weyuker, E. J. (1982) · The Computer Journal · doi:10.1093/comjnl/25.4.465 · Cited by 606 · see citing works
Golovin, D., Solnik, B., Moitra, S., Kochanski, G., Karro, J. E., & Sculley, D. (2017) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3097983.3098043 · Cited by 585 · see citing works
A Survey on Metamorphic Testing
Segura, S., Fraser, G., Sánchez, A. B., & Ruiz–Cortés, A. (2016) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2016.2532875 · Cited by 571 · see citing works
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding
Wang, A., Singh, A., Michael, J., Hill, F., Levy, O., & Bowman, S. R. (2018) · arXiv · doi:10.48550/arxiv.1804.07461 · Cited by 562 · see citing works
An empirical analysis of flaky tests
Luo, Q., Hariri, F., Eloussi, L., & Marinov, D. (2014) · SIGSOFT FSE · doi:10.1145/2635868.2635920 · Cited by 551 · see citing works
Fink, O., Wang, Q., Svensén, M., Dersin, P., Lee, W.-J., & Ducoffe, M. (2020) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2020.103678 · Cited by 549 · see citing works
(2007) · The MIT Press eBooks · doi:10.7551/mitpress/7443.001.0001 · Cited by 528 · see citing works
Adams, J., Hillier-Brown, F. C., Moore, H. J., Lake, A. A., Araujo-Soares, V., White, M., & Summerbell, C. (2016) · Systematic Reviews · doi:10.1186/s13643-016-0337-y · Cited by 486 · see citing works
Fairness-aware Learning through Regularization Approach
Kamishima, T., Akaho, S., & Sakuma, J. (2011) · 2011 IEEE 11th International Conference on Data Mining Workshops · doi:10.1109/icdmw.2011.83 · Cited by 475 · see citing works
Single data extraction generated more errors than double data extraction in systematic reviews
Buscemi, N., Hartling, L., Vandermeer, B., Tjosvold, L., & Klassen, T. P. (2006) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2005.11.010 · Cited by 464 · see citing works
Metamorphic Testing: A New Approach for Generating Next Test CasesPreprint
Chen, T. Y., Cheung, S.-C., & Yiu, S.-M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.12543 · Cited by 458 · see citing works
Guide to the Software Engineering Body of Knowledge (SWEBOK(R)): Version 3.0
Bourque, P., & Fairley, R. E. (2014) · IEEE Computer Society Press eBooks · Cited by 455 · see citing works
The Measure and Mismeasure of Fairness
Corbett-Davies, S., Gaebler, J. D., Nilforoshan, H., Shroff, R., & Goel, S. (2018) · arXiv · doi:10.48550/arxiv.1808.00023 · Cited by 444 · see citing works
DeepHunter: a coverage-guided fuzz testing framework for deep neural networks
Xie, X., Ma, L., Juefei-Xu, F., Xue, M., Chen, H., Liu, Y., Zhao, J., Li, B., Yin, J., & See, S. C. W. (2019) · ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA) · doi:10.1145/3293882.3330579 · Cited by 440 · see citing works
Fairness testing: testing software for discrimination
Galhotra, S., Brun, Y., & Meliou, A. (2017) · Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering · doi:10.1145/3106237.3106277 · Cited by 405 · see citing works
Testing and validating machine learning classifiers by metamorphic testing
Xie, X., Ho, J. W. K., Murphy, C., Kaiser, G. E., Xu, B., & Chen, T. Y. (2010) · Journal of Systems and Software · doi:10.1016/j.jss.2010.11.920 · Cited by 394 · see citing works
Establishing the rules for building trustworthy AI
Floridi, L. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0055-y · Cited by 394 · see citing works
Anonymizing Classification Data for Privacy Preservation
Fung, B. C. M., Wang, K., & Yu, P. S. (2007) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2007.1015 · Cited by 351 · see citing works
Concolic testing for deep neural networks
Sun, Y., Wu, M., Ruan, W., Huang, X., Kwiatkowska, M., & Kroening, D. (2018) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3238147.3238172 · Cited by 337 · see citing works
Learning under Concept Drift: an OverviewPreprint
Žliobaitė, I. (2010) · arXiv (Cornell University) · doi:10.48550/arxiv.1010.4784 · Cited by 332 · see citing works
On the tradeoff between privacy and utility in data publishing
Li, T., & Li, N. (2009) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/1557019.1557079 · Cited by 318 · see citing works
A comprehensive study on deep learning bug characteristics
Islam, J., Nguyen, G., Pan, R., & Rajan, H. (2019) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3338906.3338955 · Cited by 315 · see citing works
An empirical study on TensorFlow program bugs
Zhang, Y., Chen, Y., Cheung, S.-C., Xiong, Y., & Zhang, L. (2018) · ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA) · doi:10.1145/3213846.3213866 · Cited by 311 · see citing works
Naming the pain in requirements engineering
Méndez, D., Wagner, S., Kalinowski, M., Felderer, M., Mafra, P., Vetrò, A., Conte, T. U., Christiansson, M.-T., Greer, D., Lassenius, C., et al. (2016) · Empirical Software Engineering · doi:10.1007/s10664-016-9451-7 · Cited by 303 · see citing works
Taxonomy of real faults in deep learning systems
Humbatova, N., Jahangirova, G., Bavota, G., Riccio, V., Stocco, A., & Tonella, P. (2020) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3377811.3380395 · Cited by 299 · see citing works
Identifying quality-requirement conflicts
BOEHM, B. W., & In, H. P. (1996) · IEEE Software · doi:10.1109/52.506460 · Cited by 295 · see citing works
Azeem, M. I., Palomba, F., Shi, L., & Wang, Q. (2019) · Information and Software Technology · doi:10.1016/j.infsof.2018.12.009 · Cited by 295 · see citing works
DLFuzz: differential fuzzing testing of deep learning systems
Guo, J., Jiang, Y., Zhao, Y., Chen, Q., & Sun, J. (2018) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3236024.3264835 · Cited by 293 · see citing works
Efficient memory-based learning for robot control
Moore, A. (2021) · doi:10.48456/tr-209 · Cited by 284 · see citing works
Assuring the Machine Learning Lifecycle: Desiderata, Methods, and Challenges
Ashmore, R., Calinescu, R., & Paterson, C. (2022) · ACM Computing Surveys · doi:10.1145/3453444 · Cited by 256 · see citing works
Testing machine learning based systems: a systematic mapping
Riccio, V., Jahangirova, G., Stocco, A., Humbatova, N., Weiss, M., & Tonella, P. (2020) · Empirical Software Engineering · doi:10.1007/s10664-020-09881-0 · Cited by 251 · see citing works
Pospieszny, P., Czarnacka-Chrobot, B., & Kobylinski, A. (2018) · Journal of Systems and Software · doi:10.1016/j.jss.2017.11.066 · Cited by 247 · see citing works
Lwakatare, L. E., Raj, A., Crnkovic, I., Bosch, J., & Olsson, H. H. (2020) · Information and Software Technology · doi:10.1016/j.infsof.2020.106368 · Cited by 241 · see citing works
DeepGini: prioritizing massive tests to enhance the robustness of deep neural networks
Feng, Y., Shi, Q., Gao, X., Wan, J., Fang, C., & Chen, Z. (2020) · ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA) · doi:10.1145/3395363.3397357 · Cited by 230 · see citing works
Large-scale machine learning at twitter
Lin, J., & Kolcz, A. (2012) · ACM SIGMOD International Conference on Management of Data (SIGMOD) · doi:10.1145/2213836.2213958 · Cited by 221 · see citing works
Machine Learning: The High Interest Credit Card of Technical Debt
Sculley, D., Holt, G. D., Golovin, D., Davydov, E. V., Phillips, T., Ebner, D., Chaudhary, V., & Young, M. (2014) · Cited by 218 · see citing works
Data Lifecycle Challenges in Production Machine Learning: A Survey
Polyzotis, N., Roy, S., Whang, S. E., & Zinkevich, M. (2018) · ACM SIGMOD Record · doi:10.1145/3299887.3299891 · Cited by 214 · see citing works
Lwakatare, L. E., Raj, A., Bosch, J., Olsson, H. H., & Crnkovic, I. (2019) · Lecture Notes in Business Information Processing · doi:10.1007/978-3-030-19034-7_14 · Cited by 214 · see citing works
Garousi, V., Felderer, M., & Mäntylä, M. V. (2016) · Proceedings of the 20th International Conference on Evaluation and Assessment in Software Engineering · doi:10.1145/2915970.2916008 · Cited by 210 · see citing works
MODE: automated neural network model debugging via state differential analysis and input selection
Ma, S., Liu, Y., Lee, W., Zhang, X., & Grama, A. Y. (2018) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3236024.3236082 · Cited by 207 · see citing works
Machine learning applied to software testing: A systematic mapping study
Durelli, V. H. S., Durelli, R. S., Borges, S. S., Endo, A. T., Eler, M. M., Dias, D. R. C., & Guimaraes, M. P. (2019) · IEEE Transactions on Reliability · doi:10.1109/tr.2019.2892517 · Cited by 207 · see citing works
On testing machine learning programs
Braiek, H. B., & Khomh, F. (2020) · Journal of Systems and Software · doi:10.1016/j.jss.2020.110542 · Cited by 203 · see citing works
Automated directed fairness testing
Udeshi, S., Arora, P., & Chattopadhyay, S. (2018) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3238147.3238165 · Cited by 200 · see citing works
Data scientists in software teams: State of the art and challenges
Kim, M., Zimmermann, T., DeLine, R., & Begel, A. (2018) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2017.2754374 · Cited by 199 · see citing works
Alexander, I. F., & Stevens, R. (2002) · Cited by 191 · see citing works
Λουρίδας, Π., & Ebert, C. (2016) · IEEE Software · doi:10.1109/ms.2016.114 · Cited by 190 · see citing works
Dwarakanath, A., Ahuja, M., Sikand, S., et al. (2018) · ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA) · doi:10.1145/3213846.3213858 · Cited by 189 · see citing works
Machine Learning and Software Engineering
Zhang, D., & Tsai, J. J. P. (2003) · Software Quality Journal · doi:10.1023/a:1023760326768 · Cited by 184 · see citing works
When and what to automate in software testing? A multi-vocal literature review
Garousi, V., & Mäntylä, M. V. (2016) · Information and Software Technology · doi:10.1016/j.infsof.2016.04.015 · Cited by 182 · see citing works
Black box fairness testing of machine learning models
Aggarwal, A., Lohia, P., Nagar, S., Dey, K., & Saha, D. (2019) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3338906.3338937 · Cited by 180 · see citing works
Bell, J., Legunsen, O., Hilton, M. L., Eloussi, L., Yung, T., & Marinov, D. (2018) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3180155.3180164 · Cited by 170 · see citing works
"Why Software Is Eating the World"
Zuckerberg, M. (2011) · UWM Digital Commons (University of Wisconsin–Milwaukee) · Cited by 169 · see citing works
DeepStellar: model-based quantitative analysis of stateful deep learning systems
Du, X., Xie, X., Li, Y., Ma, L., Liu, Y., & Zhao, J. (2019) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3338906.3338954 · Cited by 169 · see citing works
Deep learning library testing via effective model generation
Wang, Z., Yan, M., Chen, J., Liu, S., & Zhang, D. (2020) · Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering · doi:10.1145/3368089.3409761 · Cited by 167 · see citing works
Misbehaviour prediction for autonomous driving systems
Stocco, A., Weiß, M., Calzana, M., & Tonella, P. (2020) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3377811.3380353 · Cited by 166 · see citing works
Cognitive Biases in Software Engineering: A Systematic Mapping Study
Mohanani, R., Salman, I., Turhan, B., Rodríguez, P., & Ralph, P. (2018) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2018.2877759 · Cited by 165 · see citing works
Metamorphic testing for software quality assessment: A study of search engines
Zhou, Z. Q., Xiang, S., & Chen, T. Y. (2016) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2015.2478001 · Cited by 163 · see citing works
CONSTRUE/TIS: A System for Content-Based Indexing of a Database of News Stories
Hayes, P. J., & Weinstein, S. P. (1990) · Innovative Applications of Artificial Intelligence · Cited by 159 · see citing works
Vartak, M., Subramanyam, H., Lee, W.-E., Viswanathan, S., Husnoo, S., Madden, S. R., & Zaharia, M. (2016) · Workshop on Human-In-the-Loop Data Analytics (HILDA) · doi:10.1145/2939502.2939516 · Cited by 157 · see citing works
Gade, K., Geyik, S. C., Kenthapadi, K., Mithal, V., & Taly, A. (2019) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3292500.3332281 · Cited by 156 · see citing works
Testing scientific software: A systematic literature review
Kanewala, U., & Bieman, J. M. (2014) · Information and Software Technology · doi:10.1016/j.infsof.2014.05.006 · Cited by 150 · see citing works
Is neuron coverage a meaningful measure for testing deep neural networks?
Harel-Canada, F., Wang, L., Gulzar, M. A., Gu, Q., & Kim, M. (2020) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3368089.3409754 · Cited by 150 · see citing works
Supervised Machine Learning Applied to Link Prediction in Bipartite Social Networks
Benchettara, N., Kanawati, R., & Rouveirol, C. (2010) · IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM) · doi:10.1109/asonam.2010.87 · Cited by 148 · see citing works
An Empirical Study of Model-Agnostic Techniques for Defect Prediction Models
Jiarpakdee, J., Tantithamthavorn, C. K., Dam, H. K., & Grundy, J. (2022) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2020.2982385 · Cited by 148 · see citing works
Software reuse strategies and component markets
Ravichandran, T., & Rothenberger, M. A. (2003) · Communications of the ACM · doi:10.1145/859670.859678 · Cited by 146 · see citing works
Explainable software analytics
Dam, H. K., Tran, T., & Ghose, A. (2018) · IEEE/ACM International Conference on Software Engineering: New Ideas and Emerging Results (ICSE-NIER) · doi:10.1145/3183399.3183424 · Cited by 146 · see citing works
Fairway: a way to build fair ML software
Chakraborty, J., Majumder, S., Yu, Z., & Menzies, T. (2020) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3368089.3409697 · Cited by 144 · see citing works
The Practitioner's Guide to Data Quality Improvement
(2011) · Elsevier eBooks · doi:10.1016/c2009-0-17212-4 · Cited by 143 · see citing works
Model-based exploration of the frontier of behaviours for deep learning system testing
Riccio, V., & Tonella, P. (2020) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3368089.3409730 · Cited by 141 · see citing works
A comprehensive study of autonomous vehicle bugs
Garcia, J., Feng, Y., Shen, J., et al. (2020) · doi:10.1145/3377811.3380397 · Cited by 140 · see citing works
On Challenges in Machine Learning Model Management
Schelter, S., Bießmann, F., Januschowski, T., Salinas, D., Seufert, S., & Szarvas, G. (2015) · IEEE Data(base) Engineering Bulletin · Cited by 139 · see citing works
Generating effective test cases for self-driving cars from police reports
Gambi, A., Huynh, T., & Fraser, G. (2019) · ESEC/SIGSOFT FSE · doi:10.1145/3338906.3338942 · Cited by 138 · see citing works
Zhang, J. M., Zhang, L., Harman, M., Hao, D., Jia, Y., & Zhang, L. (2018) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2018.2809496 · Cited by 137 · see citing works
Estimating GPU memory consumption of deep learning models
Gao, Y., Liu, Y., Zhang, H., Li, Z., Zhu, Y., Lin, H., & Yang, M. (2020) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3368089.3417050 · Cited by 137 · see citing works
Requirements-driven test generation for autonomous vehicles with machine learning components
Tuncali, C. E., Fainekos, G., Prokhorov, D., Ito, H., & Kapinski, J. (2020) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2019.2955903 · Cited by 129 · see citing works
Adoption and Effects of Software Engineering Best Practices in Machine Learning
Serban, A., van der Blom, K., Hoos, H., & Visser, J. (2020) · Proceedings of the 14th ACM / IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM) · doi:10.1145/3382494.3410681 · Cited by 127 · see citing works
Towards Accountable AI: Hybrid Human-Machine Analyses for Characterizing System Failure
Nushi, B., Kamar, E., & Horvitz, E. (2018) · Proceedings of the AAAI Conference on Human Computation and Crowdsourcing · doi:10.1609/hcomp.v6i1.13337 · Cited by 126 · see citing works
Neural network models for software development effort estimation: a comparative study
Nassif, A. B., Azzeh, M., Capretz, L. F., & Ho, D. (2016) · Neural Computing and Applications · doi:10.1007/s00521-015-2127-1 · Cited by 122 · see citing works
A comprehensive study on challenges in deploying deep learning based software
Chen, Z., Cao, Y., Liu, Y., Wang, H., Xie, T., & Liu, X. (2020) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3368089.3409759 · Cited by 122 · see citing works
An experience report on defect modelling in practice
Tantithamthavorn, C. K., & Hassan, A. E. (2018) · IEEE/ACM International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP) · doi:10.1145/3183519.3183547 · Cited by 121 · see citing works
Practical Software Requirements: A Manual of Content and Style
Kovitz, B. (1998) · Cited by 119 · see citing works
Aligning software engineering education with industrial needs: A meta-analysis
Garousi, V., Giray, G., Tüzün, E., Catal, C., & Felderer, M. (2019) · Journal of Systems and Software · doi:10.1016/j.jss.2019.06.044 · Cited by 119 · see citing works
Self-citationConcept drift detection for streaming data
Wang, H., & Abraham, Z. (2015) · doi:10.1109/ijcnn.2015.7280398 · Cited by 116 · see citing works
Provenance Analysis for Missing Answers and Integrity Repairs.
Zaharia, M., Chen, A., Davidson, A., Ghodsi, A., Hong, S. A., Konwinski, A., Murching, S., Nykodym, T., Ogilvie, P., Parkhe, M., et al. (2018) · IEEE Data(base) Engineering Bulletin · Cited by 112 · see citing works
Challenges in the Deployment and Operation of Machine Learning in Practice
Baier, L., Jöhren, F., & Seebacher, S. (2019) · Repository KITopen (Karlsruhe Institute of Technology) · doi:10.5445/ir/1000095028 · Cited by 110 · see citing works
Economic perspectives in test automation
Ramler, R., & Wolfmaier, K. (2006) · International Workshop on Automation of Software Test (AST) · doi:10.1145/1138929.1138946 · Cited by 106 · see citing works
ModelOps: Cloud-Based Lifecycle Management for Reliable and Trusted AI
Hummer, W., Muthusamy, V., Rausch, T., Dube, P., El Maghraoui, K., Murthi, A., & Oum, P. (2019) · 2019 IEEE International Conference on Cloud Engineering (IC2E) · doi:10.1109/ic2e.2019.00025 · Cited by 105 · see citing works
Mitchell, P. (2011) · Current Medical Research and Opinion · doi:10.1185/03007995.2011.585394 · Cited by 102 · see citing works
On challenges in engineering IoT software systems
Motta, R. C., de Oliveira, K. M., & Travassos, G. H. (2018) · Brazilian Symposium on Software Engineering · doi:10.1145/3266237.3266263 · Cited by 100 · see citing works
What Is Really Different in Engineering AI-Enabled Systems?
Özkaya, İ. (2020) · IEEE Software · doi:10.1109/ms.2020.2993662 · Cited by 100 · see citing works
Software engineering for machine-learning applications: The road ahead
Khomh, F., Adams, B., Cheng, J., Fokaefs, M., & Antoniol, G. (2018) · IEEE Software · doi:10.1109/ms.2018.3571224 · Cited by 95 · see citing works
Studying Software Engineering Patterns for Designing Machine Learning Systems
Washizaki, H., Uchida, H., Khomh, F., & Guéhéneuc, Y. (2019) · International Workshop on Empirical Software Engineering in Practice (IWESEP) · doi:10.1109/iwesep49350.2019.00017 · Cited by 94 · see citing works
Practical Accuracy Estimation for Efficient Deep Neural Network Testing
Chen, J., Wu, Z. J., Wang, Z., You, H., Zhang, L., & Yan, M. X. (2020) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3394112 · Cited by 94 · see citing works
Boosting operational DNN testing efficiency through conditioning
Li, Z., Ma, X., Xu, C., et al. (2019) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3338906.3338930 · Cited by 85 · see citing works
Engineering problems in machine learning systems
Kuwajima, H., Yasuoka, H., & Nakae, T. (2020) · Machine Learning · doi:10.1007/s10994-020-05872-w · Cited by 84 · see citing works
Creating a Software Engineering Culture
Wiegers, K. E. (1996) · Cited by 83 · see citing works
Performance and fault-tolerance of neural networks for optimization
Protzel, P., Palumbo, D., & Arras, M. K. (1993) · IEEE Transactions on Neural Networks · doi:10.1109/72.238315 · Cited by 82 · see citing works
Testing embedded software: A survey of the literature
Garousi, V., Felderer, M., Karapıçak, Ç. M., & Yılmaz, U. (2018) · Information and Software Technology · doi:10.1016/j.infsof.2018.06.016 · Cited by 81 · see citing works
Software engineering challenges for machine learning applications: A literature review
Kumeno, F. (2020) · Intelligent Decision Technologies · doi:10.3233/idt-190160 · Cited by 81 · see citing works
Detecting flaky tests in probabilistic and machine learning applications
Dutta, S., Shi, A., Choudhary, R., Zhang, Z., Jain, A., & Misailovíc, S. (2020) · ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA) · doi:10.1145/3395363.3397366 · Cited by 79 · see citing works
Correlations between deep neural network model coverage criteria and model quality
Yan, S., Tao, G., Liu, X., Zhai, J., Ma, S., Xu, L., & Zhang, X. (2020) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3368089.3409671 · Cited by 79 · see citing works
More About Software Requirements: Thorny Issues and Practical Advice
Wiegers, K. E. (2005) · Cited by 76 · see citing works
Understanding Development Process of Machine Learning Systems: Challenges and Solutions
Nascimento, E. D. S., Ahmed, I., Oliveira, E., Palheta, M., Steinmacher, I., & Conte, T. (2019) · International Symposium on Empirical Software Engineering and Measurement · doi:10.1109/esem.2019.8870157 · Cited by 72 · see citing works
Requirements Engineering in the Days of Artificial Intelligence
Dalpiaz, F., & Niu, N. (2020) · IEEE Software · doi:10.1109/ms.2020.2986047 · Cited by 72 · see citing works
A Survey of Software Quality for Machine Learning Applications
Masuda, S., Ono, K., Yasue, T., & Hosokawa, N. (2018) · IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW) · doi:10.1109/icstw.2018.00061 · Cited by 71 · see citing works
Nascimento, E., Nguyen‐Duc, A., Sundbø, I., & Conte, T. U. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.03751 · Cited by 69 · see citing works
Just Enough Software Test Automation
Mosley, D. J., & Posey, B. (2002) · International Linear Collider · Cited by 66 · see citing works
Groce, A., Kulesza, T., Zhang, C., Shamasunder, S., Burnett, M., Wong, W., Stumpf, S., Das, S., Shinsel, A., Bice, F., et al. (2014) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2013.59 · Cited by 66 · see citing works
Engineering AI Systems: A Research AgendaPreprint
Bosch, J., Crnković, I., & Olsson, H. H. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.07522 · Cited by 64 · see citing works
Peng, Z. J., Yang, J., Chen, T.-H. P., et al. (2020) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3368089.3417063 · Cited by 63 · see citing works
Artificial Intelligence and Software Engineering: Status and Future Trends.
Rech, J., & Althoff, K. (2004) · Cited by 62 · see citing works
A complexity theory viewpoint on the software development process and situational context
Clarke, P. M., O’Connor, R. V., & Leavy, B. (2016) · International Conference on Software and System Process · doi:10.1145/2904354.2904369 · Cited by 61 · see citing works
ModelHub: Deep Learning Lifecycle Management
Miao, H., Li, A., Davis, L. S., & Deshpande, A. (2017) · IEEE International Conference on Data Engineering (ICDE) · doi:10.1109/icde.2017.192 · Cited by 61 · see citing works
An Approach to Software Testing of Machine Learning Applications
Murphy, C., Kaiser, G. E., & Arias, M. (2007) · doi:10.7916/d8r49znr · Cited by 60 · see citing works
Searching for build debt: experiences managing technical debt at Google
Morgenthaler, J. D., Gridnev, M., Sauciuc, R., & Bhansali, S. (2012) · doi:10.5555/2666036.2666037 · Cited by 60 · see citing works
Testing and quality validation for AI software–perspectives, issues, and practices
Tao, C., Gao, J., & Wang, T. (2019) · IEEE Access · doi:10.1109/access.2019.2937107 · Cited by 60 · see citing works
The sensemaking-coevolution-implementation theory of software design
Ralph, P. (2015) · Science of Computer Programming · doi:10.1016/j.scico.2014.11.007 · Cited by 60 · see citing works
Towards Guidelines for Assessing Qualities of Machine Learning Systems
Siebert, J., Jöckel, L., Heidrich, J., Nakamichi, K., Ohashi, K., Namba, I., Yamamoto, R., & Aoyama, M. (2020) · Communications in computer and information science · doi:10.1007/978-3-030-58793-2_2 · Cited by 56 · see citing works
Is using deep learning frameworks free?
Liu, J., Huang, Q., Xia, X., Shihab, E., Lo, D., & Li, S. (2020) · IEEE/ACM International Conference on Software Engineering: Software Engineering in Society (ICSE-SEIS) · doi:10.1145/3377815.3381377 · Cited by 55 · see citing works
MODELDB: A System for Machine Learning Model Management.
Vartak, M. (2017) · Conference on Innovative Data Systems Research · Cited by 54 · see citing works
Teaching software engineering for AI-enabled systems
Kästner, C., & Kang, E. (2020) · IEEE/ACM International Conference on Software Engineering: Software Engineering Education and Training (ICSE-SEET) · doi:10.1145/3377814.3381714 · Cited by 53 · see citing works
Catching up with Method and Process Practice: An Industry-Informed Baseline for Researchers
Klünder, J., Hebig, R., Tell, P., Kuhrmann, M., Nabende, J. N., Heldal, R., Krusche, S., Fazal-Baqaie, M., Felderer, M., Bocco, M. G., et al. (2019) · IEEE/ACM International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP) · doi:10.1109/icse-seip.2019.00036 · Cited by 50 · see citing works
Deep Convolutional Neural Network Design PatternsPreprint
Smith, L. N., & Topin, N. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.00847 · Cited by 47 · see citing works
Tata, S., Popescul, A., Najork, M., Colagrosso, M., Gibbons, J., Green, A. I., Mah, A., Smith, M. D., Garg, D., Meyer, C., et al. (2017) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3097983.3098048 · Cited by 46 · see citing works
The Machine Learning Bazaar: Harnessing the ML Ecosystem for Effective System Development
Smith, M. J., Sala, C., Kanter, J. M., & Veeramachaneni, K. K. (2020) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 46 · see citing works
Towards a Software Engineering Process for Developing Data-Driven Applications
Hesenius, M., Schwenzfeier, N., Meyer, O., et al. (2019) · doi:10.1109/raise.2019.00014 · Cited by 45 · see citing works
Sutton, C. A., Hobson, T., Geddes, J., et al. (2018) · ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (KDD) · doi:10.1145/3219819.3220057 · Cited by 44 · see citing works
Hulten, G. (2018) · Apress eBooks · doi:10.1007/978-1-4842-3432-7 · Cited by 43 · see citing works
Menzies, T. (2020) · IEEE Software · doi:10.1109/ms.2019.2954841 · Cited by 43 · see citing works
Moreb, M., Mohammed, T. A., & Bayat, O. (2020) · IEEE Access · doi:10.1109/access.2020.2970178 · Cited by 39 · see citing works
Kang, S., FELDT, R. H., & Yoo, S. (2020) · doi:10.1145/3387940.3391456 · Cited by 39 · see citing works
Think Your Artificial Intelligence Software Is Fair? Think Again
Bellamy, R., Dey, K., Hind, M., Hoffman, S. C., Houde, S., Kannan, K., Lohia, P., Mehta, S., Mojsilović, A., Nagar, S., et al. (2019) · IEEE Software · doi:10.1109/ms.2019.2908514 · Cited by 37 · see citing works
Privacy and anonymization for very large datasets
Muntés-Mulero, V., & Nin, J. (2009) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/1645953.1646333 · Cited by 32 · see citing works
The Machine Learning Bazaar: Harnessing the ML Ecosystem for Effective System Development
Smith, M. J., Sala, C., Kanter, J. M., et al. (2020) · Proceedings - ACM-SIGMOD International Conference on Management of Data · doi:10.1145/3318464.3386146 · Cited by 31 · see citing works
Machine Learning Software Engineering in Practice: An Industrial Case StudyPreprint
Rahman, M. S., Rivera, E., Khomh, F., Guéhéneuc, Y., & Lehnert, B. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.07154 · Cited by 31 · see citing works
Embrace the challenges: software engineering in a big data world
Anderson, K. M. (2015) · International Workshop on BIG Data Software Engineering · doi:10.5555/2819289.2819297 · Cited by 30 · see citing works
Jung, J. (2005) · Cited by 28 · see citing works
Bridging the gap between ML solutions and their business requirements using feature interactions
Barash, G., Farchi, E., Jayaraman, I., et al. (2019) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3338906.3340442 · Cited by 28 · see citing works
CERSE - Catalog for empirical research in software engineering: A Systematic mapping study
Molléri, J. S., Petersen, K., & Mendes, E. (2018) · Information and Software Technology · doi:10.1016/j.infsof.2018.08.008 · Cited by 26 · see citing works
On Testing Data-Intensive Software Systems
Felderer, M., Russo, B., & Auer, F. (2019) · Springer eBooks · doi:10.1007/978-3-030-25312-7_6 · Cited by 26 · see citing works
Detecting and understanding real-world differential performance bugs in machine learning libraries
Tizpaz-Niari, S., Černý, P., & Trivedi, A. (2020) · ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA) · doi:10.1145/3395363.3404540 · Cited by 25 · see citing works
Operational calibration: debugging confidence errors for DNNs in the field
Li, Z., Ma, X., Xu, C., Xu, J., Cao, C., & Lü, J. (2020) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3368089.3409696 · Cited by 24 · see citing works
Emerging and Changing Tasks in the Development Process for Machine Learning Systems
Liu, H., Eksmo, S., Risberg, J., & Hebig, R. (2020) · International Conference on Software and System Process · doi:10.1145/3379177.3388905 · Cited by 24 · see citing works
On the Use of Ontologies in Software Process Assessment
Tarhan, A., & Giray, G. (2017) · International Conference on Evaluation and Assessment in Software Engineering (EASE) · doi:10.1145/3084226.3084261 · Cited by 21 · see citing works
Self-citation
Flagged first, then by citations
Research works printed but not linked: 6
- Fujii, 2020, Guidelines for quality assurance of machine learning-based artificial intelligence, Int. J. Softw. Eng. Knowl. Eng., 1
- Wan, 2019, How does machine learning change software development practices?, IEEE Trans. Softw. Eng.
- Fei-Fei, L., 2010. ImageNet: crowdsourcing, benchmarking & other cool things. In: CMU VASC Seminar (Vol. 16). March. pp. 18-25.
- Schwerdtner, P., Grezner, F., Kapoor, N., Assion, F., Sass, R., Günther, W., Hüger, F., Schlicht, P., 2020. Risk assessment for machine learning models. In: Machine Learning for Autonomous Driving Workshop at the 34th Conference on Neural Information Processing Systems (NeurIPS 2020), Vancouver, Canada.
- Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I.J., Fergus, R., 2014. Intriguing properties of neural networks. In: 2nd International Conference on Learning Representations (ICLR).
- Zaharia, 2018, Accelerating the machine learning lifecycle with mlflow, IEEE Data Eng. Bull., 41, 39
Cited by
Sorted by citations
Operationalizing machine learning models: a systematic literature review
Kolltveit, A. B., & Li, J. (2022) · Proceedings of the 1st Workshop on Software Engineering for Responsible AI · doi:10.1145/3526073.3527584 · Cited by 34 · see citing works
- Authors
- Giray, G.
- Corresponding author
- Giray, Görkem
- ORCID
- Giray, Görkem
- Venue
- Journal of Systems and Software
- Publisher
- Elsevier BV
- Published
- 2021-10
- Volume
- 180
- Pages
- 111031
- ISSN
- 0164-1212 (ISSN-L)
- OpenAlex
- W3112475255
- Cited by
- 187
- References
- 256
- Citations by year
- 2026: 23 · 2025: 37 · 2024: 38 · 2023: 43 · 2022: 36 · 2021: 10
- FWCI
- 28.71
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- Journal of Systems and Software (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by-sa)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Computer Science › Artificial Intelligence › Advanced Software Engineering Methodologies
- Keywords
- software engineering for machine learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-10.
Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-10.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
