Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat
Sign Up
Cited by 19

The DOI matches its Crossref record; 3 of the 3 references it cites were found in a registry or an index.

The Evolution of Technical Debt from DevOps to Generative AI: A multivocal literature review

Moreschini, Sergio, Arvanitou, Elvira-Maria, Kanidou, Elisavet-Persefoni, Nikolaidis, Nikolaos, Su, Ruoyu, Ampatzoglou, Apostolos, Chatzigeorgiou, Alexander, Lenarduzzi, Valentina·2026·Journal of Systems and Software
DOI 10.1016/j.jss.2025.112599
Open this paper in CoChat — free account

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1016/j.jss.2025.112599

Cites

3 of 3 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 3 references

Research works printed but not linked: 51

  1. Albuquerque, 2023, Managing technical debt using intelligent techniques - a systematic mapping study, IEEE Trans. Softw. Eng., 49, 2202, 10.1109/TSE.2022.3214764 · DOI
  2. Alfayez, R., Alwehaibi, W., Winn, R., Venson, E., Boehm, B., 2020. A Systematic Literature Review of Technical Debt Prioritization. In: IEEE/ACM International Conference on Technical Debt (TechDebt). Seoul, Korea. · DOI
  3. Alves, 2016, Identification and management of technical debt: A systematic mapping study, Inf. Softw. Technol., 70, 10.1016/j.infsof.2015.10.008 · DOI · Paper page
  4. Ampatzoglou, 2015, The financial aspect of managing technical debt: A systematic literature review, Inf. Softw. Technol., 64, 52, 10.1016/j.infsof.2015.04.001 · DOI
  5. Becker, C., Chitchyan, R., Betz, S., McCord, C., 2018. Trade-Off Decisions across Time in Technical Debt Management: A Systematic Literature Review. In: 2018 IEEE/ACM International Conference on Technical Debt (TechDebt). Gothenburg, Sweden, pp. 85–94. · DOI
  6. Besker, 2018, Managing architectural technical debt: A unified model and systematic literature review, J. Syst. Softw., 135, 10.1016/j.jss.2017.09.025 · DOI
  7. Bhandari, K., Kumar, K., Sangal, A.L., 2023. Artificial Intelligence in Software Engineering: Perspectives and Challenges. In: 2023 Third International Conference on Secure Cyber Computing and Communication. ICSCCC, pp. 133–137. · DOI
  8. Bogner, J., Verdecchia, R., Gerostathopoulos, I., 2021. Characterizing Technical Debt and Antipatterns in AI-Based Systems: A Systematic Mapping Study. In: 2021 IEEE/ACM International Conference on Technical Debt (TechDebt). pp. 64–73. · DOI
  9. Durrani, 2024, A decade of progress: A systematic literature review on the integration of AI in software engineering phases and activities (2013–2023), IEEE Access, 12, 171185, 10.1109/ACCESS.2024.3488904 · DOI
  10. Fernández-Sánchez, 2017, Identification and analysis of the elements required to manage technical debt by means of a systematic mapping study, J. Syst. Softw., 124, 10.1016/j.jss.2016.10.018 · DOI
  11. Junior, 2022, Consolidating a common perspective on technical debt and its management through a tertiary study, Inf. Softw. Technol., 149, 10.1016/j.infsof.2022.106964 · DOI
  12. Khomyakov, Z., Makhmutov, Z., Mirgalimova, R., Sillitti, A., 2019. Automated Measurement of Technical Debt: A Systematic Literature Review. In: 21st International Conference on Enterprise Information Systems (ICEIS 2019). Crete, Greece, pp. 95–106. · DOI
  13. Kitchenham, 2013, A systematic review of systematic review process research in software engineering, Inf. Softw. Technol., 55, 2049, 10.1016/j.infsof.2013.07.010 · DOI · Paper page
  14. Kleinwaks, 2023, Technical debt in systems engineering—A systematic literature review, Syst. Eng., 26, 675, 10.1002/sys.21681 · DOI
  15. Klimczyk, 2020, Technical debt aware estimations in software engineering: A systematic mapping study, E-Inform. Softw. Eng. J., 14, 61
  16. Koulla Moulla, D., Mnkandla, E., Oumarou, H., Fehlmann, T., 2024. Technical Debt Measurement: An Exploratory Literature Review. In: 33rd International Workshop on Software Measurement and 18th International Conference on Software Process and Product Measurement (IWSM-MENSURA’24). Montréal, Canada.
  17. Lenarduzzi, 2021, A systematic literature review on technical debt prioritization: Strategies, processes, factors, and tools, J. Syst. Softw., 171, 10.1016/j.jss.2020.110827 · DOI
  18. Li, 2015, A systematic mapping study on technical debt and its management, J. Syst. Softw., 101, 10.1016/j.jss.2014.12.027 · DOI · Paper page
  19. Li, 2022, OSSARA: abandonment risk assessment for embedded open source components, IEEE Softw., 39, 48, 10.1109/MS.2022.3163011 · DOI
  20. Lunde, B.A., Colomo-Palacios, R., 2020. Continuous practices and technical debt: a systematic literature review. In: 2020 20th International Conference on Computational Science and Its Applications. ICCSA, Cagliari, Italy, pp. 40–44. · DOI
  21. McDermott, 2020, AI4SE and SE4AI: A research roadmap, INSIGHT, 23, 47, 10.1002/inst.12278 · DOI
  22. Melo, 2022, Identification and measurement of requirements technical debt in software development: A systematic literature review, J. Syst. Softw., 194, 10.1016/j.jss.2022.111483 · DOI
  23. Moreschini, S., Lomio, F., Hästbacka, D., Taibi, D., 2022. MLOps for evolvable AI intensive software systems. In: 2022 IEEE International Conference on Software Analysis, Evolution and Reengineering. SANER, pp. 1293–1294. · DOI
  24. Nielsen, 2020, Technical debt management: A systematic literature review and research agenda for digital government, vol. 12219
  25. Perera, J., Tempero, E., Tu, Y.-C., Blincoe, K., 2023. Quantifying Requirements Technical Debt: A Systematic Mapping Study and a Conceptual Model. In: 31st International Requirements Engineering Conference. RE, Hannover, Germany. · DOI
  26. Perera, 2024, A systematic mapping study exploring quantification approaches to code, design, and architecture technical debt, ACM Trans. Softw. Eng. Methodol., 33, 1, 10.1145/3675393 · DOI
  27. Ribeiro, L.F., Farias, M.A.d.F., Mendonça, M., Spínola, R.O., 2016. Decision Criteria for the Payment of Technical Debt in Software Projects: A Systematic Mapping Study. In: Proceedings of the 18th International Conference on Enterprise Information Systems (ICEIS 2016). Setubal, Portugal. · DOI
  28. Rios, 2018, A tertiary study on technical debt: Types, management strategies, research trends, and base information for practitioners, Inf. Softw. Technol., 102, 117, 10.1016/j.infsof.2018.05.010 · DOI
  29. Ruiz, 2024, A tertiary study on technical debt management over the last lustrum, Int. J. Comb. Optim. Probl. Inform., 15, 181
  30. Saeeda, 2024, A multivocal literature review on non-technical debt in software development: An insight into process, social, people, organizational, and culture debt, E-Inform. Softw. Eng. J., 18
  31. da Silva, 2022, Exploring technical debt tools: A systematic mapping study, vol. 455
  32. Sofian, 2022, Systematic mapping: Artificial intelligence techniques in software engineering, IEEE Access, 10, 51021, 10.1109/ACCESS.2022.3174115 · DOI
  33. Sousa, L., Rocha, L., Britto, R., 2023. Architectural Technical Debt - A Systematic Mapping Study. In: XXXVII Brazilian Symposium on Software Engineering. SBES’23, Campo Grande, Brazil. · DOI
  34. Villa, J., Ocharán-Hernández, J.O., Pérez-Arriaga, J.C., Limón, X., 2022. A Systematic Mapping Study on Technical Debt in Microservices. In: 10th International Conference in Software Engineering Research and Innovation. CONISOFT, Ciudad Modelo, San José Chiapa, Mexico. · DOI
  35. Baruzzo, A., Gabbrielli, M., The Pragmatic Programmer for Machine Learning, Chapter 5: Designing and Structuring Pipelines. Available at: https://ppml.dev/design-code.html.
  36. Chaudhary, 2018, A review on hidden debts in machine learning systems, 619
  37. Chen, K.-H., Su, H.-P., Chuang, W.-C., Hsiao, H.-C., Tan, W., Tang, Z., Liu, X., Liang, Y., Lo, W.-C., Ji, W., et al., 2022. Apache submarine: A unified machine learning platform made simple. In: Proceedings of the 2nd European Workshop on Machine Learning and Systems. pp. 101–108. · DOI
  38. Codabux, 2021, Technical debt in the peer-review documentation of r packages: A rOpenSci case study, 195
  39. Foidl, H., Felderer, M., Ramler, R., 2022. Data smells: Categories, causes and consequences, and detection of suspicious data in AI-based systems. In: Proceedings of the 1st International Conference on AI Engineering: Software Engineering for AI. pp. 229–239. · DOI
  40. Fu, Y., Wang, T., Li, S., Ding, J., Zhou, S., Jia, Z., Li, W., Jiang, Y., Liao, X., 2024. MissConf: LLM-Enhanced Reproduction of Configuration-Triggered Bugs. In: Proceedings of the 2024 IEEE/ACM 46th International Conference on Software Engineering: Companion Proceedings. pp. 484–495. · DOI
  41. Li, 2022, Identifying self-admitted technical debt in issue tracking systems using machine learning, Empir. Softw. Eng., 27, 131, 10.1007/s10664-022-10128-3 · DOI
  42. Menshawy, A., Nawaz, Z., Fahmy, M., 2024. Navigating Challenges and Technical Debt in Large Language Models Deployment. In: Proceedings of the 4th Workshop on Machine Learning and Systems. pp. 192–199. · DOI
  43. Moreschini, S., Lenarduzzi, V., Coba, L., 2024. Towards a Technical Debt for AI-based Recommender System. In: Proceedings of the 7th ACM/IEEE International Conference on Technical Debt. pp. 36–39. · DOI
  44. Raffin, 2022, A reference architecture for the operationalization of machine learning models in manufacturing, Procedia CIRP, 115, 130, 10.1016/j.procir.2022.10.062 · DOI
  45. Recupito, 2024, Technical debt in AI-enabled systems: On the prevalence, severity, impact, and management strategies for code and architecture, J. Syst. Softw., 216, 10.1016/j.jss.2024.112151 · DOI
  46. Sculley, D., Holt, G., Golovin, D., Davydov, E., Phillips, T., Ebner, D., Chaudhary, V., Young, M., 2014. Machine learning: The high interest credit card of technical debt. In: SE4ML: Software Engineering for Machine Learning. NIPS 2014 Workshop, Cambridge, MA, p. 112.
  47. Sculley, 2015, Hidden technical debt in machine learning systems, Adv. Neural Inf. Process. Syst., 28
  48. Sharma, 2022, Self-admitted technical debt in R: detection and causes, Autom. Softw. Eng., 29, 53, 10.1007/s10515-022-00358-6 · DOI
  49. Shome, A., Cruz, L., Van Deursen, A., 2022. Data smells in public datasets. In: Proceedings of the 1st International Conference on AI Engineering: Software Engineering for AI. pp. 205–216. · DOI
  50. Sklavenitis, D., Kalles, D., 2024. Measuring Technical Debt in AI-Based Competition Platforms. In: Proceedings of the 13th Hellenic Conference on Artificial Intelligence. pp. 1–10. · DOI
  51. Zhang, H., Cruz, L., Van Deursen, A., 2022. Code smells for machine learning applications. In: Proceedings of the 1st International Conference on AI Engineering: Software Engineering for AI. pp. 217–228. · DOI

Cited by

Metadata

Authors
Moreschini, S., Arvanitou, E.-M., Kanidou, E.-P., Nikolaidis, N., Su, R., Ampatzoglou, A., Chatzigeorgiou, A., & Lenarduzzi, V.
ORCID
Venue
Journal of Systems and Software
Publisher
Elsevier BV
Published
2026-01
Volume
231
Pages
112599
ISSN
0164-1212
DOI
10.1016/j.jss.2025.112599
Cited by
19
References
3
Language
en
Licence
Funders
  • Business Finland
  • Horizon Europe (101058479)

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.