The five laws of SE for AI
Abstract
It is time to talk about software engineering (SE) for artificial intelligence (AI). As shown in Figure 1, industry is becoming increasingly dependent on AI software. Clearly, AI is useful for SE. But what about the other way around? How important is SE for AI? Many thought leaders in the AI industry are asking how to better develop and maintain AI software (see Figure 2).
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Tripakis (2018) and Menzies (2019) points out the heterogeneity of DL platforms that leads to interoperability problems.
“Tripakis (2018) and Menzies (2019) points out the heterogeneity of DL platforms that leads to interoperability problems.”
From A software engineering perspective on engineering machine learning systems: State of the art and challenges (2021) · cites this paper as Menzies, 2019
Cites
Scikit-learn: Machine Learning in Python
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Müller, A., Nothman, J., Louppe, G., et al. (2012) · arXiv · doi:10.48550/arxiv.1201.0490 · Cited by 71,016 · see citing works
Software Engineering for Machine Learning: A Case Study
Amershi, S., Begel, A., Bird, C., DeLine, R. A., Gall, H. C., Kamar, E. S., Nagappan, N., Nushi, B., & Zimmermann, T. (2019) · IEEE/ACM International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP) · doi:10.1109/icse-seip.2019.00042 · Cited by 1,181 · see citing works
(2015) · neural information processing systems · Cited by 912 · see citing works
Automated parameter optimization of classification techniques for defect prediction models
Tantithamthavorn, C. K., McIntosh, S., Hassan, A. E., & Matsumoto, K. (2016) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/2884781.2884857 · Cited by 333 · see citing works
Testing vision-based control systems using learnable evolutionary algorithms
Abdessalem, R. B., Nejati, S., Briand, L., et al. (2018) · Proceedings/Proceedings - International Conference on Software Engineering · doi:10.1145/3180155.3180160 · Cited by 269 · see citing works
On the "naturalness" of buggy code
Ray, B., Hellendoorn, V. J., Godhane, S., Tu, Z., Bacchelli, A., & Dévanbu, P. (2016) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/2884781.2884848 · Cited by 260 · see citing works
A benchmark study on sentiment analysis for software engineering research
Novielli, N., Girardi, D., & Lanubile, F. (2018) · International Conference on Mining Software Repositories (MSR) · doi:10.1145/3196398.3196403 · Cited by 97 · see citing works
Deploying Search Based Software Engineering with Sapienz at Facebook
Alshahwan, N., Gao, X., Harman, M., et al. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-99241-9_1 · Cited by 77 · see citing works
AI‐Based Software Defect Predictors: Applications and Benefits in a Case Study
Tosun, A., Bener, A., & Kale, R. (2011) · AI Magazine · doi:10.1609/aimag.v32i2.2348 · Cited by 67 · see citing works
CODEMINE: Building a Software Development Data Analytics Platform at Microsoft
Czerwonka, J., Nagappan, N., Schulte, W., et al. (2013) · IEEE Software · doi:10.1109/ms.2013.68 · Cited by 67 · see citing works
The open-closed principle of modern machine learning frameworks
Braiek, H. B., Khomh, F., & Adams, B. (2018) · International Conference on Mining Software Repositories (MSR) · doi:10.1145/3196398.3196445 · Cited by 39 · see citing works
Backes, J., Bolignano, P., Cook, B., Gacek, A., Luckow, K. S., Rungta, N., Schäf, M., Schlesinger, C., Tanash, R. S., Varming, C., et al. (2019) · IEEE Software · doi:10.1109/ms.2019.2930609 · Cited by 31 · see citing works
The need for software specific natural language techniques
Binkley, D., Lawrie, D., & Morrell, C. H. (2017) · Empirical Software Engineering · doi:10.1007/s10664-017-9566-5 · Cited by 20 · see citing works
Krall, J., Menzies, T., & Davies, M. (2016) · IEEE Transactions on Human-Machine Systems · doi:10.1109/thms.2015.2509980 · Cited by 11 · see citing works
Self-citationIs AI different for SEPreprint
Agrawal, A., & Menzies, T. (2019) · arXiv (Cornell University) · Cited by 4 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
“Everyone wants to do the model work, not the data work”: Data Cascades in High-Stakes AI
Sambasivan, N., Kapania, S., Highfill, H., Akrong, D., Paritosh, P., & Aroyo, L. M. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445518 · Cited by 781 · see citing works
Giray, G. (2021) · Journal of Systems and Software · doi:10.1016/j.jss.2021.111031 · Cited by 187 · see citing works
- Authors
- Menzies, T.
- Affiliations
- North Carolina State University (US)
- Corresponding author
- Menzies, Tim
- ORCID
- Menzies, Tim
- Venue
- IEEE Software
- Published
- 2020-01
- Volume
- 37
- Issue
- 1
- Pages
- 81-85
- ISSN
- 0740-7459 (ISSN-L)
- 1937-4194
- OpenAlex
- W2995639860
- Cited by
- 43
- References
- 15
- Citations by year
- 2025: 7 · 2024: 3 · 2023: 6 · 2022: 6 · 2021: 10 · 2020: 11
- FWCI
- 8.39
- Type
- journal article
- Language
- it
- Open access
- closed
- Licence
- Locations
- IEEE Software (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Computer Science › Artificial Intelligence › Advanced Software Engineering Methodologies
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
