A 2030 Roadmap for Software Engineering
Open accessThe DOI matches its Crossref record; 244 of the 244 references it cites were found in a registry or an index; 16 of the 149 sources its claims cite were checked for support.
Abstract
The landscape of software engineering has dramatically changed in recent years. The impressive advances of artificial intelligence are just the latest and most disruptive innovation that has remarkably changed the software engineering research and practice. This special issue shares a roadmap to guide the software engineering community in this confused era. This roadmap is the outcome of a 2-day intensive discussion at the 2030 Software Engineering workshop. The roadmap spotlights and discusses seven main landmarks in the new software engineering landscape: artificial intelligence for software engineering, human aspects of software engineering, software security, verification and validation, sustainable software engineering, automatic programming, and quantum software engineering. This editorial summarizes the core aspects discussed in the 37 papers that comprise the seven sections of the special issue and guides the interested readers throughout the issue. This roadmap is a living body that we will refine with follow-up workshops that will update the roadmap for a series of forthcoming ACM TOSEM special issues.
Overview
CoChat Summary
TL;DR
This editorial/roadmap from the 2030 Software Engineering workshop identifies seven landmarks shaping software engineering toward 2030, centered on generative AI’s disruptive impact. It summarizes 37 papers across seven sections and proposes open research directions for AI, human aspects, sustainability, automatic programming, security, verification/validation, and quantum software engineering.
Key findings
- The roadmap spotlights seven main areas: artificial intelligence for software engineering, human aspects of software engineering, software security, verification and validation, sustainable software engineering, automatic programming, and quantum software engineering.
- The workshop was a 2-day event co-located with ACM SIGSOFT FSE on 15–16 July 2024 in Porto De Galinhas, Brazil, with 62 participants; 58 of 76 submitted papers were invited to extend, and the special issue ultimately includes 33 selected papers plus 4 editors’ papers, totaling 37 papers.
- Generative AI is described as triggering “the deepest change in the skyline of software engineering research and practice since the Internet revolution,” shifting developers toward orchestrating, validating, and curating AI-generated artifacts.
- The roadmap emphasizes challenges such as unbiased datasets and benchmarks, reliable prompts, explainability, ethics, trust in automatically generated code, security of AI-generated code, verification of evolving human-centric ecosystems, and quantum software engineering’s need for new lifecycle approaches.
Methods
- Design: editorial/roadmap produced from intensive 2-day discussions at the 2030 Software Engineering Workshop, co-located with ACM SIGSOFT FSE on 15–16 July 2024 in Porto De Galinhas, Brazil, using liberating structures such as impromptu networking, flash keynotes, 1-2-4-All, world café, fishbowl, open space technology, and conversation café.
- Data/sample: 62 workshop participants; 58 of 76 workshop submissions were invited for extension; after ACM TOSEM peer review, 33 papers were selected for the special issue, plus 4 editors’ papers, for 37 papers across seven sections.
- Analysis: qualitative synthesis organized around seven research areas; each section overviews state of the art/practice, discusses challenges and trends, and proposes a roadmap, using McLuhan’s tetrads to illustrate disruptive impacts.
Limitations
- The paper states the roadmap is “a living body” to be refined with follow-up workshops and forthcoming ACM TOSEM special issues, and that “any roadmap quickly ages.”
- The paper states that the quantum software engineering community’s limited access to quantum hardware may limit innovations in that area.
- Evident from the text: this is an editorial/roadmap synthesizing workshop discussions and 37 papers, not a primary empirical study or systematic review with a formal data-collection and analysis protocol; no quantitative evaluation of the roadmap’s outcomes is reported.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
11 Valid · 6 Not valid · 79 Not checked · 105 Own results and interpretations
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“The successful use of zero-shot and one-shot learning to summarize the code [229, 235] and of conversational prompts for automatic program repair [222-224, 230] leads to new efficient prompting strategies in software engineering.”
What the sources say
- Not checked yetcites [229]
- Not checked yetcites [235]
- Not checked yetcites [222]
- Not checked yetcites [223]
- Not checked yetcites [224]
- Supportedcites [230] source
The citation to CREF is attached to the conversational-prompting part of the sentence, not the code-summarization part. Section 4.3, “Conversational Program Repair” (page unavailable), reports that CREF uses historical conversational entries and improves repair performance over the baseline. The broader conclusion about new prompting strategies is the citing authors’ synthesis. (LLM, conf=0.91)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“The inadequate evaluation of deep learning systems can have significant consequences, including risks to patient well-being [165] and safety [208].”
What the sources say
- Partly supportedcites [165] sourceExperimental — accuracy not yet measured
Our copy of Richens et al. (Introduction) does state that deficiencies in associative diagnostic algorithms — the class that includes Deep Learning approaches — carry consequences for patients: 'failure to disentangle correlation from causation places strong constraints on the accuracy of associative diagnostic algorithms, sometimes resulting in sub-optimal or dangerous diagnoses', and the Discussion (Diagnostic model and datasets) flags limits of validation on EHR data (mislabeled ground truth). So the 'risk to patient well-being/safety from deficient deep learning systems' part is in the paper. What the paper does NOT say is that these risks stem from inadequate EVALUATION of deep learning systems: its argument is about inadequate inference scheme (associative vs counterfactual), and its remarks on evaluation concern the difficulty of defining ground truth, not patient harm. The serious-harm figure it does give ('one in three of these misdiagnoses resulting in serious patient harm', Introduction) is credited to Singh/Graber diagnostic-error studies (its refs 1,4), not to deep learning evaluation. (LLM, conf=0.65)
- Could not verifycites [208] sourceExperimental — accuracy not yet measured
Our copy of Wang et al. (2020) is a review of autonomous-vehicle safety; it nowhere addresses deep learning systems or their evaluation, and its safety consequences concern road users (drivers, pedestrians, cyclists), not patients. Searched: 'deep learning evaluation of neural network models' (only reference-list entries about LSTM-GAN/machine-learning control appear), 'patient well-being and safety risks in healthcare', 'insufficient testing of algorithms, training data, model validation', and 'consequences of system failure for human health' — the closest passages (3.1.1 Perception Error, 3.1.2 Decision Error, 5.2.2 Minimizing Decision Errors) discuss sensor/software/decision errors and the need for 'extensive and strict hardware and software testing' in AVs only. Since no passage speaks to the claim's point, the attribution can be neither confirmed nor contradicted on our copy. (LLM, conf=0.72)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
None of the sources cited for this claim have been checked yet.
In the paper
“Few studies propose new automated testing methods to address the diverse scenarios of deep learning models in software engineering [70, 195, 225].”
What the sources say
- Not checked yetcites [70]
- Not checked yetcites [195]
- Not checked yetcites [225]
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“Current studies that explore the use of artificial intelligence for software engineering research remain in their early stages [226].”
What the sources say
- Not checked yetcites [226]
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“Current metrics are limited in evaluating accuracy, precision, recall, and perplexity [37, 110, 111, 227].”
What the sources say
- Not checked yetcites [37]
- Not checked yetcites [110]
- Not checked yetcites [111]
- Not checked yetcites [227]
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
27 more claims
Create a free account to see all 201 claims and to verify the 133 references still open.
- Probably not a substantial claim. 149
- It may state two separate claims as one. 83
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ 6.1 State of the Art and Trends · p. 32
“The editors' paper [117] in this special issue provides a comprehensive review of the literature on the use of large language models for detecting and preparing vulnerabilities.”
§ 1 Introduction · p. 3
“These papers address four hot topics: Artificial Intelligence for Software Engineering [33], Software Engineering by and for Humans [1], Automatic Programming [117], and Software Security Analysis [26].”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Overview of the Editorial and the Special Issue · p. 7
“Murillo et al.'s [134] and Ramalho et al.'s [162] papers that comprise Section 8 of this special issue”
§ 8.1.1 Requirements Engineering. · p. 39
“Murillo et al.'s paper [133] in this special issue briefly mentions that quantum requirements engineering will inevitably deviate from classic requirement engineering approaches.”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ 3.1 State of the Art and Practice · p. 18
“propose six interconnected themes:”
§ 3.1 State of the Art and Practice · p. 18
“individual capabilities, team capabilities, product, social impact, and human aspects.”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ 4 Sustainable Software Engineering · p. 22
“the 1972 Brundtland 5 report of the UN World Commission on Environment and Development [24]”
§ References · p. 45
“[24] Gro Harlem Brundtland. 1987. Our Common Future World Commission on Environment and Developement . UN World Commission on Environment and Development.”
Citations
Cites
Items 201–244 of 244
Requirements Are All You Need: The Final Frontier for End-User Software Engineering
Robinson, D., Cabrera, C., Gordon, A. D., Lawrence, N. D., & Mennen, L. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3708524 · Cited by 12
VeriContaminated: Assessing LLM-Driven Verilog Coding for Data ContaminationPreprint
Wang, Z., Shao, M., Bhandari, J., Mankali, L., Karri, R., Sinanoglu, O., Shafique, M., & Knechtel, J. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2503.13572 · Cited by 12
Which Syntactic Capabilities Are Statistically Learned by Masked Language Models for Code?
Velasco, A., Palacio, D. N., Rodríguez-Cárdenas, D., & Poshyvanyk, D. (2024) · IEEE/ACM International Conference on Software Engineering: New Ideas and Emerging Results (ICSE-NIER) · doi:10.1145/3639476.3639768 · Cited by 11
Self-citationMorescient GAI for Software Engineering
Kessel, M., & Atkinson, C. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3709354 · Cited by 11
Penzenstadler, B. (2017) · Information and Software Technology · doi:10.1016/j.infsof.2017.11.011 · Cited by 10
Self-citationInnovating for Tomorrow: The Convergence of SE and Green AIPreprint
Cruz, L., Gutiérrez, X. F., & Martínez-Fernández, S. (2024) · arXiv.org · doi:10.48550/arxiv.2406.18142 · Cited by 9
Toward Neurosymbolic Program Comprehension
Velasco, A., Garryyeva, A., Palacio, D. N., Mastropaolo, A., & Poshyvanyk, D. (2025) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1109/icpc66645.2025.00047 · Cited by 9
Self-citationPredicting How to Test Requirements: An Automated Approach
Winkler, J., Grönberg, J., & Vogelsang, A. (2019) · IEEE International Requirements Engineering Conference (RE) · doi:10.1109/re.2019.00023 · Cited by 8
Iterative Sustainability Impact Assessment: When to propose?
Penzenstadler, B., Betz, S., Duboc, L., Seyff, N., Porras, J., Oyedeji, S., Brooks, I., & Venters, C. C. (2021) · 2021 IEEE/ACM International Workshop on Body of Knowledge for Software Sustainability (BoKSS) · doi:10.1109/bokss52540.2021.00010 · Cited by 8
Self-citationInterpretation of Learning-Based Automatic Source Code Vulnerability Detection Model Using LIME
Tang, G., Zhang, L., Yang, F., Meng, L., Cao, W., Qiu, M., Ren, S., Yang, L., & Wang, H. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-82153-1_23 · Cited by 7
Interpretable Software Defect Prediction Incorporating Multiple Rules
Yang, F., Zeng, G., Zhong, F., Zheng, W., & Xiao, P. (2023) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner56733.2023.00114 · Cited by 7
Interpretable Software Maintenance and Support Effort Prediction Using Machine Learning
Haldar, S., & Capretz, L. F. (2024) · IEEE/ACM International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3639478.3643069 · Cited by 7
Marshall McLuhan’s General Theory of Media (GToM), His Laws of Media; Comparing Three Kinds of LawPreprint
Logan, R. K. (2021) · Preprints.org · doi:10.20944/preprints202104.0526.v1 · Cited by 6
Moussa, N. E., Molinelli, D., Pezzè, M., & Tappler, M. (2021) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3468264.3473137 · Cited by 6
Self-citationSustainability in Computing Education: A Systematic Literature Review
Peters, A.-K., Capilla, R., Coroamă, V. C., Heldal, R., Lago, P., Leifler, O., Moreira, A., Fernandes, J. P., Penzenstadler, B., Porras, J., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2305.10369 · Cited by 6
Self-citationUsing Transfer Learning for Code-Related TasksPreprint
Mastropaolo, A., Cooper, N., Palacio, D. N., et al. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2206.08574 · Cited by 5
Counterfactual Explanations for Models of CodePreprint
Cito, J., Dillig, I., Murali, V., & Chandra, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.05711 · Cited by 5
Shi, J., Yang, Z., & Lo, D. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2404.04566 · Cited by 4
Automatic Programming: Large Language Models and BeyondPreprint
Lyu, M. R.-T., Ray, B., Roychoudhury, A., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2405.02213 · Cited by 4
Self-citationBeyond Accuracy and Robustness Metrics for Large Language Models for Code
Rodríguez-Cárdenas, D. (2024) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3639478.3639792 · Cited by 4
Explainable AI for Pre-Trained Code Models: What Do They Learn? When They Do Not Work?Preprint
Mohammadkhani, A. H., Tantithamthavorn, C. K., & Hemmati, H. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2211.12821 · Cited by 4
Unveiling Code Pre-Trained Models: Investigating Syntax and Semantics CapacitiesPreprint
Ma, W., Liu, S., Zhao, M., et al. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2212.10017 · Cited by 4
Open-Source AI-based SE Tools: Opportunities and Challenges of Collaborative Software LearningPreprint
Lin, Z., Ma, W., Lin, T., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2404.06201 · Cited by 3
The Current Challenges of Software Engineering in the Era of Large Language ModelsPreprint
Gao, C., Hu, X., Gao, S., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2412.14554 · Cited by 3
How Propense Are Large Language Models at Producing Code Smells? A Benchmarking StudyPreprint
Velasco, A., Rodríguez-Cárdenas, D., Alif, L. R., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2412.18989 · Cited by 3
Engineering Digital Systems for Humanity: a Research RoadmapPreprint
Autili, M., Sanctis, M. D., Inverardi, P., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2412.19668 · Cited by 3
Stalnaker, T., Wintersgill, N., Chaparro, O., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2502.04484 · Cited by 3
Birchler, C., Khatiri, S., Rani, P., Kehrer, T., & Panichella, S. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2405.01064 · Cited by 2
Testing and Debugging Quantum Programs: The Road to 2030Preprint
Ramalho, N. C. L., de Souza, H. A., & Chaim, M. L. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2405.09178 · Cited by 2
Requirements are All You Need: The Final Frontier for End-User Software EngineeringPreprint
Robinson, D., Cabrera, C., Gordon, A. D., Lawrence, N. D., & Mennen, L. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2405.13708 · Cited by 2
With Great Power Comes Great Responsibility: The Role of Software EngineersPreprint
Betz, S., & Penzenstadler, B. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2407.08823 · Cited by 2
Self-citationToward Neurosymbolic Program ComprehensionPreprint
Velasco, A., Garryyeva, A., Palacio, D. N., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2502.01806 · Cited by 2
Investigating Poor Performance Regions of Black Boxes: LIME-based Exploration in Sepsis DetectionPreprint
Salimiparsa, M., Parmar, S., Lee, S., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2306.12507 · Cited by 2
Agentic AI Software Engineers: Programming with TrustPreprint
Roychoudhury, A., Păsăreanu, C. S., Pradel, M., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2502.13767 · Cited by 2
Self-citationTowards Generating Executable Metamorphic Relations Using Large Language ModelsPreprint
Shin, S. Y., Pastore, F., Bianculli, D., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2401.17019 · Cited by 1
Foundation Model Engineering: Engineering Foundation Models Just as Engineering SoftwarePreprint
Ran, D., Wu, M. S., Yang, W. M., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2407.08176 · Cited by 1
The Flexcrash Platform for Testing Autonomous Vehicles in Mixed-Traffic Scenarios
Gambi, A., Mathews, S., Steininger, B., Poienko, M., & Bobek, D. (2024) · Proceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis · doi:10.1145/3650212.3685299 · Cited by 1
Software Security Analysis in 2030 and Beyond: A Research RoadmapPreprint
Böhme, M., Bodden, E., Bultan, T., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2409.17844 · Cited by 1
Improving Requirements Completeness: Automated Assistance through Large Language ModelsPreprint
Luitel, D., Hassani, S., & Sabetzadeh, M. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2308.03784 · Cited by 1
On the Reliability and Explainability of Language Models for Program GenerationPreprint
Liu, Y., Tantithamthavorn, C. K., Liu, Y., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2302.09587
The Sustainability Assessment Framework Toolkit: A Decade of Modeling ExperiencePreprint
Lago, P., Condorí-Fernández, N., Fatima, I., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2405.01391
Automated Trustworthiness Oracle Generation for Machine Learning Text ClassifiersPreprint
Tung, L. N., Cho, S. B., Du, X., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2410.22663
Prompt Alchemy: Automatic Prompt Refinement for Enhancing Code GenerationPreprint
Ye, S., Sun, Z., Wang, G., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2503.11085
Mapping the Trust Terrain: LLMs in Software Engineering -- Insights and PerspectivesPreprint
Khati, D., Liu, Y., Palacio, D. N., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2503.13793
Research works printed but not linked: 23
- Gro Harlem Brundtland. 1987. Our Common Future World Commission on Environment and Developement. UN World Commission on Environment and Development.
- Johan Cederbladh, Antonio Cicchetti, and Robbert Jongeling. n.d. A road-map to readily available early validation & verification of system behaviour in model-based systems engineering using software engineering best practices. ACM Transaction on Software Engineering and Methodology 2030 Roadmap Special Issue, 1049-331X.
- Xinyun Chen, Chang Liu, and Dawn Song.2018. Execution-guided neural program synthesis. In Proceedings of the International Conference on Learning Representations.
- Eliot Zackrone, Beiza Eken, Michal Ernst, Mauro Pezzè, Davide Molinelli, and Alberto Martin-Lopez. 2025. Tratto: A neuro-symbolic approach to deriving axiomatic test oracles. In Proceedings of the ACM ISSTA International Symposium on Software Testing and Analysis.
- Elizabeth Dinella, Gabriel Ryan, Todd Mytkowicz, and Shuvendu Lahiri. 2022. TOGA: A neural method for test oracle generation. In Proceedings of the International Conference on Software Engineering (ICSE ’22). ACM. Retrieved from https://www.microsoft.com/en-us/research/publication/toga-a-neural-method-for-test-oracle-generation/
- Guoxiang Guo Aldeida Aleti Neelofar Neelofar and Chakkrit Tantithamthavorn. 2024. MORTAR: Metamorphic multi-turn testing for LLM-based dialogue systems. arXiv:2412.15557. Retrieved from https://arxiv.org/abs/2412.15557 · arXiv
- Ashish Hooda, Mihai Christodorescu, Miltiadis Allamanis, Aaron Wilson, Kassem Fawaz, and Somesh Jha. 2024. Do large code models understand programming concepts? Counterfactual analysis for code predicates. In Proceedings of the 41st International Conference on Machine Learning (ICML’24). Article 753, 11 pages.
- Sonja Hyrynsalmi, Sebastian Baltes, Chris Brown, Rafael Prikladnicki, Perez Gema Rodriguez, Alexander Serebrenik, Jocelyn Simmonds, Bianca Trinkenreich, and Yi Wang. 2025. Bridging gaps, building futures: Advancing software developer diversity and inclusion through future-oriented research. ACM Transaction on Software Engineering and Methodology 2030 Roadmap Special Issue, 1049-331X.
- Aditya Kanade, Petros Maniatis, Gogul Balakrishnan, and Kensen Shi. 2020. Learning and evaluating contextual embedding of source code. In Proceedings of the International Conference on Machine Learning. PMLR, 5110–5121.
- Fatemeh Khayashi Behnaz Jamasb Reza Akbari and Pirooz Shamsinejadbabaki. 2022. Deep learning methods for software requirement classification: A performance study on the PURE dataset. arXiv:2211.05286. Retrieved from https://arxiv.org/abs/2211.05286 · arXiv
- Jian Li Yue Wang Michael R. Lyu and Irwin King. 2017. Code completion with neural attention and pointer networks. arXiv:1711.09573. Retrieved from https://arxiv.org/abs/1711.09573 · arXiv
- Ruijie Meng, Martin Mirchev, Marcel Böhme, and Abhik Roychoudhury. 2024. Large language model guided protocol fuzzing. In Proceedings of the 31st Annual Network and Distributed System Security Symposium (NDSS).
- Juan Manuel Murillo, Jose Garcia-Alonso, Enrique Moguel, Johanna Barzen, Frank Leymann, Shaukat Ali, Tao Yue, Paolo Arcaini, Ricardo Perez-Castillo, Ignacio García-Rodríguez de Guzmán, et al. 2025. Quantum software engineering: Roadmap and challenges ahead. ACM Transaction on Software Engineering and Methodology 2030 Roadmap Special Issue, 1049-331X.
- Birgit Penzenstadler. 2014. Infusing green: Requirements engineering for green In and through software systems. In Proceedings of the International Workshop on Requirements Engineering for Sustainable Systems (RE4SuSy@ RE), 44–53.
- Jari Porras, Colin C. Venters, Birgit Penzenstadler, Leticia Duboc, Stefanie Betz, Norbert Seyff, Saeid Heshmatisafa, and Shola Oyedeji. 2021. How could we have known? Anticipating sustainability effects of a software product. In Proceedings of the 12th International Conference Software Business (ICSOB ’21). Springer, 10–17.
- J. Schneider-Hufschmidt, T. Kühme, and U. Malinowski. 1993. Adaptive User Interfaces: Principles and Practice. North Holland, London.
- Norbert Seyff, Stefanie Betz, Leticia Duboc, Colin Venters, Christoph Becker, Ruzanna Chitchyan, Birgit Penzenstadler, and Markus Nöbauer. 2018. Tailoring requirements negotiation to sustainability. In Proceedings of the 2018 IEEE 26th International Requirements Engineering Conference (RE). IEEE, 304–314.
- Trevor Stalnaker Nathan Wintersgill Oscar Chaparro Laura A. Heymann Massimiliano Di Penta Daniel M. German and Denys Poshyvanyk. 2024. Developer perspectives on licensing and copyright issues arising from generative AI for coding. arXiv:2411.10877. Retrieved from https://arxiv.org/abs/2411.10877 · arXiv
- Colin C. Venters, Caroline Jay, L. M. S. Lau, Michael K. Griffiths, Violeta Holmes, Rupert R. Ward, Jim Austin, Charlie E. Dibsdale, and Jie Xu. 2014. Software sustainability: The modern tower of Babel. In Proceedings of the CEUR Workshop 1216 (2014), 7–12.
- Mingxuan Xiao Yan Xiao Shunhui Ji Yunhe Li Lei Xue and Pengcheng Zhang. 2025. ABFS: Natural robustness testing for LLM-based NLP software. arXiv:2503.01319. Retrieved from https://arxiv.org/abs/2503.01319 · arXiv
- Junjielong Xu, Qinan Zhang, Zhiqing Zhong, Shilin He, Chaoyun Zhang, Qingwei Lin, Dan Pei, Pinjia He, Dongmei Zhang, and Qi Zhang. 2025. OpenRCA: Can large language models locate the root cause of software failures? In Proceedings of the 13th International Conference on Learning Representations (ICLR).
- Tao Yue, Wolfgang Mauerer, Shaukat Ali, and Davide Taibi. 2023. Challenges and opportunities in quantum software architecture. Software Architecture: Research Roadmaps from the Community, 1–23.
- 10.1109/tse.2018.2887384 · DOI · Paper page
Cited by
Amalfitano, D., Metzger, A., Autili, M., Fulcini, T., Hey, T., Keim, J., Pelliccione, P., Scotti, V., Koziolek, A., Mirandola, R., et al. (2026) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3788879 · Cited by 30
Rani, P. J. E., Gopinath, K., Deepika, G., Priyadharshini, K., Dhamodaran, S., & Mohanraj, B. (2025) · International Conference on Sustainable Communication Networks and Application (ICSCNA) · doi:10.1109/icscn67106.2025.11308225 · Cited by 6
Do LLMs Generate Useful Test Oracles? An Empirical Study with an Unbiased Dataset
Molinelli, D., Grazia, L. D., Martin-Lopez, A., Ernst, M. D., & Pezzè, M. (2025) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase63991.2025.00031 · Cited by 16
Self-citationSantos, S. D. O., Alves, R. D. M., Oliveira, F., & Araujo, F. I. S. (2025) · Brazilian Symposium on Multimedia and the Web · doi:10.5753/webmedia.2025.16160 · Cited by 4
Migliarini, P., Autili, M., Inverardi, P., & Pelliccione, P. (2026) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3801980 · Cited by 1
Metadata
- Authors
- Pezzè, M., Abrahão, S., Penzenstadler, B., Poshyvanyk, D., Roychoudhury, A., & Yue, T.
- Affiliations
- Constructor Institute (CH)
- Universitat Politècnica de València (ES)
- Lappeenranta-Lahti University of Technology (FI)
- William & Mary (US)
- National University of Singapore (SG)
- Beihang University (CN)
- Venue
- ACM Transactions on Software Engineering and Methodology
- Published
- 2025-05-26
- Volume
- 34
- Issue
- 5
- Pages
- 1-55
- ISSN
- 1049-331X (ISSN-L)
- 1557-7392
- OpenAlex
- W4409797578
- Semantic Scholar
- 14ecf35dcc975a5f5ed5e0cbaf38cc7bbfe4fc54
- Cited by
- 13
- References
- 196
- Citations by year
- 2026: 10 · 2025: 3
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- ACM Transactions on Software Engineering and Methodology (publishedVersion, cc-by)
- Chalmers Research (Chalmers University of Technology) (submittedVersion)
- Funders
- Swiss National Foundation (SNF 200021_215487)
- State Research Agency (PID2022-140106NB-I00)
- Generalitat Valenciana (CIAICO/2021/303 (AKILA))
- Denys Poshyvanyk’s research (NSF CCF-234635, CCF-2311469, CNS-2132281, and CCF-1955853)
- Singapore Ministry of Education (MoE) Tier3 (MOE-MOET32021-0001)
- State Key Laboratory of Complex & Critical Software Environment (CCSE-2024ZX-01)
- Fundamental Research Funds for the Central Universities
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Computer Science › Software › Advanced Software Engineering Methodologies
- Keywords
- human aspects of software engineering, software security, verification and validation, sustainable software development, program synthesis, quantum software engineering, future of software engineering
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-05, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-05 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol and qwen3.8-omni-flash).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.