Cited by 71Software Engineering Techniques and PracticesArtificial Intelligence ApplicationsEthics and Social Impacts of AI

AI system engineering key challenges and lessons learned

Open access
Fischer, Lukas, Ehrlinger, Lisa, Geist, Verena, Ramler, Rudolf, Sobiezky, Florian, Zellinger, Werner, Brunner, David, Kumar, Mohit +1 more·2020·Machine Learning and Knowledge Extraction

Abstract

The main challenges are discussed together with the lessons learned from past and ongoing research along the development cycle of machine learning systems. This will be done by taking into account intrinsic conditions of nowadays deep learning models, data and software quality issues and human-centered artificial intelligence (AI) postulates, including confidentiality and ethical aspects. The analysis outlines a fundamental theory-practice gap which superimposes the challenges of AI system engineering at the level of data quality assurance, model building, software engineering and deployment. The aim of this paper is to pinpoint research topics to explore approaches to address these challenges.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.3390/make3010004

Cites

172 of 173 references verified0 citation errors · 1 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 9

  1. Holzinger, A., Kieseberg, P., Tjoa, A.M., Weippl, E.R., Eds.; Springer: Berlin/Heidelberg, Germany, 2020; Volume 12279, pp. 451-471.
  2. Vapnik, V.N. Statistical Learning Theory; Wiley-Interscience: Hoboken, NJ, USA, 1998.
  3. Gal, Y. Uncertainty in Deep Learning. Ph.D. Thesis, University of Cambridge, Cambridge, UK, 2016.
  4. Chaudhuri, K., Salakhutdinov, R., Eds.; PMLR: Long Beach, CA, USA, 2019; Volume 97, pp. 6438-6447.
  5. Rigaki, M.; Garcia, S. A Survey of Privacy Attacks in Machine Learning. arXiv 2020, arXiv:2007.07646. · arXiv
  6. De La Prieta, F., Mathieu, P., Rincón Arango, J.A., El Bolock, A., Del Val, E., Jordán Prunera, J., Carneiro, J., Fuentes, R., Lopes, F., Julian, V., Eds.; Springer International Publishing: Cham, Switzerland, 2020; pp. 42-53.
  7. Kersting, K., Ed.; TU Darmstadt, Darmstadt, Germany, 2020; Volume 34, pp. 193-198.
  8. Navab, N.; Hornegger, J.; Wells, W.M.; Frangi, A.F., Eds.; Springer: Cham, Switzerland, 2015; pp. 234-241. 120.
  9. He, K.; Gkioxari, G.; Dollár, P.; Girshick, R. Mask R-CNN. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), Venice, Italy, 22-29 October 2017. 121.

Cited by

Sorted by citations

Authors
Fischer, L., Ehrlinger, L., Geist, V., Ramler, R., Sobiezky, F., Zellinger, W., Brunner, D., Kumar, M., & Moser, B.
Affiliations
  • Software Competence Center Hagenberg (Austria) (AT)
  • Johannes Kepler University of Linz (AT)
  • Software Competence Center Hagenberg GmbH (SCCH), 4232 Hagenberg, Austria
Corresponding author
Fischer, Lukas
Venue
Machine Learning and Knowledge Extraction
Publisher
MDPI AG
Published
2020-12-31
Volume
3
Issue
1
Pages
56-83
ISSN
2504-4990 (ISSN-L)
OpenAlex
W3117442988
Cited by
71
References
163
Citations by year
2026: 20 · 2025: 12 · 2024: 16 · 2023: 14 · 2022: 6 · 2021: 3
FWCI
5.41
Type
journal article
Language
en
Open access
Locations
  • Machine Learning and Knowledge Extraction (publishedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
  • Bundesministerium für Klimaschutz, Umwelt, Energie, Mobilität, Innovation und Technologie
  • Österreichische Forschungsförderungsgesellschaft
  • Bundesministerium für Digitalisierung und Wirtschaftsstandort
Topics
  • Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
  • Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence Applications
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
Keywords
machine learning systems, deep learning models, data quality assurance, software quality, human-centered AI, AI ethics, confidentiality, model deployment, theory-practice gap

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review