Requirements Are All You Need: The Final Frontier for End-User Software Engineering
Open accessThe DOI matches its Crossref record; 76 of the 77 references it cites were found in a registry or an index, and 1 are not checked yet.
Abstract
What if end-users could own the software development lifecycle from conception to deployment using only requirements expressed in language, images, video or audio? We explore this idea, building on the capabilities that Generative AI brings to software generation and maintenance techniques. How could designing software in this way better serve end-users? What are the implications of this process for the future of end-user software engineering and the software development lifecycle? We discuss the research needed to bridge the gap between where we are today and these imagined systems of the future.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“End-user programming agents are giving rise to democratized software development [110].”
From The Rise of Agentic AI: A Review of Definitions, Frameworks, Architectures, Applications, Evaluation Metrics, and Challenges (2025) · cites this paper as [110]
In the citing paper
“LLM models support problem-solving [137] and programming [155] by completing or correcting code, and describing runtime errors [110].”
From The Rise of Agentic AI: A Review of Definitions, Frameworks, Architectures, Applications, Evaluation Metrics, and Challenges (2025) · cites this paper as [110]
In the citing paper
“Automated Test Generation [110] is a method in which tools like CodeT, Reflexion, and ClarifyGPT are used to refine code and detect ambiguous requirements.”
From The Rise of Agentic AI: A Review of Definitions, Frameworks, Architectures, Applications, Evaluation Metrics, and Challenges (2025) · cites this paper as [110]
Citations
Cites
The weirdest people in the world?
Henrich, J., Heine, S. J., & Norenzayan, A. (2010) · Behavioral and Brain Sciences · doi:10.1017/s0140525x0999152x · Cited by 12,641
I.—COMPUTING MACHINERY AND INTELLIGENCE
TURING, A. M. (1950) · Mind · doi:10.1093/mind/lix.236.433 · Cited by 10,290 · cited 1×
“Gray [37] described automatic programming as a Turing-style imitation game for a programming staff [99]: replace a software development team with a computer that is ‘better and requires no more time than dealing with a typical human staff’ [37].”
Energy and Policy Considerations for Deep Learning in NLP
Strubell, E., Ganesh, A., & McCallum, A. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1355 · Cited by 3,684 · cited 1×
“For example, end-user technological constraints can be expressed as factual knowledge that LLMs responses must respect.”
Guidelines for Human-AI Interaction
Amershi, S., Weld, D., Vorvoreanu, M., Fourney, A., Nushi, B., Collisson, P., Suh, J., Iqbal, S., Bennett, P. N., Inkpen, K., et al. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300233 · Cited by 2,439 · cited 1×
“Responsible AI research has led to large companies issuing guidelines and checklists [3] while researchers examine their applicability in practice [78].”
Feuerriegel, S., Hartmann, J., Janiesch, C., & Zschech, P. (2024) · Business & Information Systems Engineering · doi:10.1007/s12599-023-00834-7 · Cited by 2,267
Reflexion: Language Agents with Verbal Reinforcement Learning
Shinn, N., Cassano, F., Berman, E., Gopinath, A., Narasimhan, K., & Yao, S. (2023) · arXiv · doi:10.48550/arxiv.2303.11366 · Cited by 1,087 · cited 1×
“…from natural language descriptions to effectively select appropriate code from amongst the many possibilities suggested by an LLM. Reflexion [85] achieved state of the art performance on HumanEval using a prompt chaining method that generates code, applies tests, reflects on error messages…”
(2019) · Cited by 1,082
Watch what I do: programming by demonstration
Cypher, A., Halbert, D. C., Kurlander, D., Lieberman, H. A., Maulsby, D., Myers, B. A., & Turransky, A. (1993) · Medical Entomology and Zoology · Cited by 1,027
Gemini: A Family of Highly Capable Multimodal Models
Gemini Team, Anil, R., Borgeaud, S., Alayrac, J.-B., Yu, J., Soricut, R., Schalkwyk, J., Dai, A. M., Hauth, A., Millican, K., et al. (2023) · arXiv · doi:10.48550/arxiv.2312.11805 · Cited by 990
Large language models for software engineering: survey and open problems
Fan, A., Gokkaya, B., Harman, M., Lyubarskiy, M., Sengupta, S., Yoo, S., & Zhang, J. M. (2023) · 2023 IEEE/ACM International Conference on Software Engineering: Future of Software Engineering (ICSE-FoSE) · doi:10.1109/icse-fose59343.2023.00008 · Cited by 650
Challenges in deploying machine learning: A survey of case studies
Paleyes, A., Urma, R.-G., & Lawrence, N. D. (2023) · ACM Computing Surveys · doi:10.1145/3533378 · Cited by 638
Self-citationThe state of the art in end-user software engineering
Ko, A. J., Abraham, R., Beckwith, L., Blackwell, A., Burnett, M., Erwig, M., Scaffidi, C., Lawrance, J., Lieberman, H., Myers, B., et al. (2011) · ACM Computing Surveys · doi:10.1145/1922649.1922658 · Cited by 633
Authentication in distributed systems: theory and practice
Lampson, B., Abadi, M., Burrows, M., & Wobber, E. (1992) · ACM Transactions on Computer Systems · doi:10.1145/138873.138874 · Cited by 607
Six Learning Barriers in End-User Programming Systems
Ko, A. J., Myers, B. A., & Aung, H. H. (2004) · 2004 IEEE Symposium on Visual Languages - Human Centric Computing · doi:10.1109/vlhcc.2004.47 · Cited by 549 · cited 1×
“Here are the requirements for an app to track the currency requirements 7 of an airline’s roster of pilots.”
Large Language Models for Software Engineering: Survey and Open Problems
Fan, A., Gokkaya, B., Harman, M., Lyubarskiy, M., Sengupta, S., Yoo, S., & Zhang, J. M. (2023) · arXiv · doi:10.48550/arxiv.2310.03533 · Cited by 523
The Metacognitive Demands and Opportunities of Generative AI
Tankelevitch, L., Kewenig, V., Simkute, A., Scott, A. E., Sarkar, A., Sellen, A., & Rintel, S. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642902 · Cited by 493
Barrera, D., Kayacik, H. G., van Oorschot, P. C., & Somayaji, A. (2010) · Proceedings of the 17th ACM conference on Computer and communications security · doi:10.1145/1866307.1866317 · Cited by 485 · cited 1×
“These tools make details visible [11], entice the user [80], utilise natural problem solving interactions like asking why questions [53], provide user interfaces to inspect formulas [31] and help users think about details in context [24] (perhaps via simulation or prototyping of a narrow slice of…”
ToolQA: A Dataset for LLM Question Answering with External Tools
Zhuang, Y., Yu, Y., Wang, K., Sun, H., & Zhang, C. (2023) · neural information processing systems · doi:10.52202/075280-2180 · Cited by 465
Benchmarking Large Language Models in Retrieval-Augmented Generation
Chen, J., Lin, H., Han, X., & Sun, L. (2023) · arXiv · doi:10.48550/arxiv.2309.01431 · Cited by 455
Benchmarking Large Language Models in Retrieval-Augmented Generation
Chen, J., Lin, H., Han, X., & Sun, L. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i16.29728 · Cited by 454
ToolQA: A Dataset for LLM Question Answering with External ToolsPreprint
Zhuang, Y.-C., Yu, Y., Wang, K., Sun, H., & Zhang, C. (2023) · Neural Information Processing Systems · doi:10.48550/arxiv.2306.13304 · Cited by 434
A Small Matter of Programming: Perspectives on End User Computing
Nardi, B. A. (1993) · The MIT Press eBooks · doi:10.7551/mitpress/1020.001.0001 · Cited by 430
The Lean Theorem Prover (System Description)
de Moura, L., Kong, S., Avigad, J., van Doorn, F., & von Raumer, J. (2015) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-21401-6_26 · Cited by 416 · cited 1×
“Shumailov et al. found that over time models trained on LLM-generated data would skew from the underlying learning task [86].”
Designing the whyline: a debugging interface for asking questions about program behavior
Ko, A. J., & Myers, B. A. (2004) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/985692.985712 · Cited by 361
GenderMag: A Method for Evaluating Software's Gender Inclusiveness
Burnett, M., Stumpf, S., Macbeth, J., Makri, S., Beckwith, L., Kwan, I., Peters, A., & Jernigan, W. (2016) · Interacting with Computers · doi:10.1093/iwc/iwv046 · Cited by 293 · cited 1×
“Similarly, mounting evidence shows that some usability issues affect women and men differently [12, 68].”
Spreadsheet data manipulation using examples
Gulwani, S., Harris, W. R., & Singh, R. (2012) · Communications of the ACM · doi:10.1145/2240236.2240260 · Cited by 292 · cited 1×
“Following years of research on programming by example [25], FlashFill, released in Excel 2013 [39], enabled widespread use.”
Barricelli, B. R., Cassano, F., Fogli, D., & Piccinno, A. (2019) · Journal of Systems and Software · doi:10.1016/j.jss.2018.11.041 · Cited by 265 · cited 1×
“Around that time, researchers identified and studied end-user programmers , professionals in some area other than software engineering who nonetheless develop skills in programming to get work done in their domain of expertise [6, 69], such as spreadsheet users.”
Understanding accountability in algorithmic supply chains
Cobbe, J., Veale, M., & Singh, J. (2023) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3593013.3594073 · Cited by 255
Optimizing Existing Software With Genetic Programming
Langdon, W. B., & Harman, M. (2015) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2013.2281544 · Cited by 233
Burnett, M., Cook, C., & Rothermel, G. (2004) · Communications of the ACM · doi:10.1145/1015864.1015889 · Cited by 215
Technology readiness levels for machine learning systems
Lavin, A., Gilligan-Lee, C. M., Visnjic, A., Ganju, S., Newman, D., Ganguly, S., Lange, D., Baydin, A. G., Sharma, A., Gibson, A., et al. (2022) · Nature Communications · doi:10.1038/s41467-022-33128-9 · Cited by 209
Ozkaya, I. (2023) · IEEE Software · doi:10.1109/ms.2023.3248401 · Cited by 206 · cited 3×
“In the example, they may include access privileges to the system, such as pilots not having write access to flight logs and takeoffs.”
Systems Engineering: Fundamentals and Applications
Haberfellner, R., Weck, O. L. D., Fricke, E., & Vössner, S. (2019) · Springer eBooks · doi:10.1007/978-3-030-13431-0 · Cited by 201
Autonomic Computing: Principles, Design and Implementation
Lalanda, P., McCann, J. A., & Diaconescu, A. (2013) · Undergraduate topics in computer science · doi:10.1007/978-1-4471-5007-7 · Cited by 180
Software Engineering of Self-adaptive Systems
Weyns, D. (2019) · Springer eBooks · doi:10.1007/978-3-030-00262-6_11 · Cited by 161 · cited 1×
“Autonomous agents use this knowledge to make adaptive decisions [14, 102].”
Secure distributed programming with value-dependent types
Swamy, N., Chen, J., Fournet, C., Strub, P.-Y., Bhargavan, K., & Yang, J. (2011) · ACM SIGPLAN Notices · doi:10.1145/2034574.2034811 · Cited by 156
Fakih, M., Dharmaji, R., Moghaddas, Y., Quirós, G., Ogundare, O., & Faruque, M. A. A. (2024) · IEEE/ACM International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP) · doi:10.1145/3639477.3639743 · Cited by 150
A Survey of Machine Learning for Big Code and Naturalness
Allamanis, M., Barr, E. T., Devanbu, P., & Sutton, C. (2019) · ACM Computing Surveys · doi:10.1145/3212695 · Cited by 150 · cited 1×
“This capability has been helpful in code generation, which can support several areas of software engineering [2, 15, 30, 72].”
Auditing Algorithms: Understanding Algorithmic Systems from the Outside In
Metaxa, D., Park, J. S., Robertson, R. E., Karahalios, K., Wilson, C., Hancock, J., & Sandvig, C. (2021) · Foundations and Trends® in Human–Computer Interaction · doi:10.1561/1100000083 · Cited by 140 · cited 1×
“In addition, algorithmic audit is a ‘method of repeatedly querying an algorithm and observing its output to analyse the algorithm’s opaque inner workings and possible external impact’ [65].”
Measuring GitHub Copilot's Impact on Productivity
Ziegler, A., Kalliamvakou, E., Li, X. A., Rice, A., Rifkin, D., Simister, S., Sittampalam, G., & Aftandilian, E. (2024) · Communications of the ACM · doi:10.1145/3633453 · Cited by 124 · cited 1×
“GitHub’s Copilot, launched in mid-2021, empowers professional developers by using an LLM to generate snippets of code from textual prompts, to explain code, to diagnose errors and to repair them [106].”
Dynamic Service Placement in Multi-Access Edge Computing: A Systematic Literature Review
Malazi, H. T., Chaudhry, S. R., Kazmi, A., Palade, A., Cabrera, C., White, G., & Clarke, S. (2022) · IEEE Access · doi:10.1109/access.2022.3160738 · Cited by 122
Self-citationLiu, M. X., Sarkar, A., Negreanu, C., Zorn, B., Williams, J., Toronto, N., & Gordon, A. D. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580817 · Cited by 119 · cited 1×
“Following the success of GitHub Copilot since 2021 in empowering developers with code autocomplete, researchers adapted code-generating LLMs to generate spreadsheet calculations [62], with eventual impact in Excel Copilot.”
Self-citationSoK: Lessons Learned from Android Security Research for Appified Software Platforms
Acar, Y., Backes, M., Bugiel, S., Fahl, S., McDaniel, P. D., & Smith, M. (2016) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp.2016.33 · Cited by 113 · cited 1×
“…select appropriate code from amongst the many possibilities suggested by an LLM. Reflexion [85] achieved state of the art performance on HumanEval using a prompt chaining method that generates code, applies tests, reflects on error messages produced and then repairs the errors discovered.”
Decision Provenance: Harnessing Data Flow for Accountable Systems
Singh, J., Cobbe, J., & Norval, C. (2019) · IEEE Access · doi:10.1109/access.2018.2887201 · Cited by 110
What next? A dozen information-technology research goals
Gray, J. (2003) · Journal of the ACM · doi:10.1145/602382.602401 · Cited by 105 · cited 3×
“Automatic Programming (from Gray’s Turing Award Lecture [37]): An Imitation Game for a Programming Staff Devise a specification language or UI (1) That is easy for people to express designs (1,000 × easier), (2) That computers can compile, and (3) That can describe all applications (is complete).”
CodeT: Code Generation with Generated Tests
Chen, B., Zhang, F., Nguyen, A., Zan, D., Lin, Z., Lou, J.-G., & Chen, W. (2022) · arXiv · doi:10.48550/arxiv.2207.10397 · Cited by 92 · cited 1×
“Her ‘Good-enough Realm’ is the bottom left quadrant, where the consequences of failure are low and the degree of human oversight is high.”
Ironies of Generative AI: Understanding and Mitigating Productivity Loss in Human-AI Interaction
Simkute, A., Tankelevitch, L., Kewenig, V., Scott, A. E., Sellen, A., & Rintel, S. (2024) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2024.2405782 · Cited by 89 · cited 1×
“Here are the requirements for an app to track the currency requirements 7 of an airline’s roster of pilots.”
Empowering End Users in Debugging Trigger-Action Rules
Corno, F., Russis, L. D., & Roffarello, A. M. (2019) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300618 · Cited by 75
Interactive thesaurus navigation: Intelligence rules OK?
Jones, S., Gatford, M., Robertson, S., Hancock‐Beaulieu, M., Secker, J., & Walker, S. (1995) · Journal of the American Society for Information Science · doi:10.1002/(sici)1097-4571(199501)46:1<9::aid-asi2>3.0.co;2-3 · Cited by 73
What Is End-User Software Engineering and Why Does It Matter?
Burnett, M. (2009) · Lecture notes in computer science · doi:10.1007/978-3-642-00427-8_2 · Cited by 62 · cited 2×
“As we move from a paradigm that involves many skilled humans in the pipeline to one in which LLMs are mediating the interaction between end-users and the software produced, it is important to consider the domains in which this sort of experimentation would be responsible.”
Sammet, J. Ε. (1978) · ACM eBooks · doi:10.1145/800025.1198367 · Cited by 59
Vaithilingam, P., Arawjo, I., & Glassman, E. L. (2024) · Designing Interactive Systems Conference · doi:10.1145/3643834.3661525 · Cited by 55 · cited 1×
“Here are the requirements for an app to track the currency requirements 7 of an airline’s roster of pilots.”
On the foolishness of “natural language programming”
Dijkstra, E. W. (1979) · Lecture notes in computer science · doi:10.1007/bfb0014656 · Cited by 54 · cited 1×
“They must perform at least three takeoffs and landings within the last 90 days to carry passengers.”
What is it like to program with artificial intelligence?
Sarkar, A., Gordon, A. D., Negreanu, C., Poelitz, C., Ragavan, S. S., & Zorn, B. (2022) · arXiv · doi:10.48550/arxiv.2208.06213 · Cited by 47 · cited 1×
“Alongside natural language, there is a great opportunity for end-users to draw images or make short videos to express requirements.”
Self-citationClover: Closed-Loop Verifiable Code Generation
Sun, C., Sheng, Y., Padon, O., & Barrett, C. (2024) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-65112-0_7 · Cited by 43
Clover: Closed-Loop Verifiable Code Generation
Sun, C., Sheng, Y., Padon, O., & Barrett, C. (2023) · arXiv · doi:10.48550/arxiv.2310.17807 · Cited by 43 · cited 2×
“Alongside natural language, there is a great opportunity for end-users to draw images or make short videos to express requirements.”
28th International Conference on Intelligent User Interfaces
(2023) · doi:10.1145/3581754 · Cited by 42
College of Engineering, T. T. U., Yelamarthi, K., Dandu, R., Mechanical Engineering Technology, K. S. U., Rao, M., Department of Computer Science, T. W. U., Yanambaka, V. P., College of Engineering, T. T. U., Mahajan, S., & College of Engineering, T. T. U. (2024) · Journal of Engineering Education Transformations · doi:10.16920/jeet/2024/v37is2/24072 · Cited by 39
Towards AI-Assisted Synthesis of Verified Dafny Methods
Misu, M. R. H., Lopes, C. V., Ma, I., & Noble, J. (2024) · Proceedings of the ACM on Software Engineering · doi:10.1145/3643763 · Cited by 38
A small matter of programming perspectives on end user computing
Marchionini, G. (1995) · Journal of the American Society for Information Science · doi:10.1002/(sici)1097-4571(199501)46:1<78::aid-asi12>3.0.co;2-p · Cited by 38
XSA: eXplainable Self-Adaptation
Camilli, M., Mirandola, R., & Scandurra, P. (2022) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3551349.3559552 · Cited by 29
Rewarding "Good" Behavior: End-User Debugging and Rewards
Ruthruff, J. R., Phalgune, A., Beckwith, L., Burnett, M., & Cook, C. (2005) · IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) · doi:10.1109/vlhcc.2004.44 · Cited by 27 · cited 1×
“Hence, the outputs of requirements-driven EUSE tools should be contained to particular domains in the way that Khlaaf suggests in her work on Operational Design Domains (ODDs) [49].”
MAACO: A Dynamic Service Placement Model for Smart Cities
Jojoa, C. H. C., Svorobej, S., Palade, A., Kazmi, A., & Clarke, S. (2022) · IEEE Transactions on Services Computing · doi:10.1109/tsc.2022.3143029 · Cited by 27 · cited 1×
“Prompt engineering or Retrieval-Augmented Generation [21] are promising research directions [15] to include external data and make LLMs’ outputs more consistent.”
COLDECO: An End User Spreadsheet Inspection Tool for AI-Generated Code
Ferdowsi, K., Williams, J. M., Drosos, I., Gordon, A. D., Negreanu, C., Polikarpova, N., Sarkar, A., & Zorn, B. G. (2023) · IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) · doi:10.1109/vl-hcc57772.2023.00017 · Cited by 26 · cited 5×
“Software Components Synthesised by LLMs. LLM hallucinations can cause application misbehaviour by injecting errors into software components, deployment plans and maintenance decisions.”
InfluentialSelf-citationLeadership of urban digital innovation for public value: A competency framework
Bastidas, V., Oti‐Sarpong, K., Nochta, T., Wan, L., Tang, J., & Schooling, J. M. (2023) · IET Smart Cities · doi:10.1049/smc2.12063 · Cited by 24 · cited 1×
“The first research area focuses on redefining competencies frameworks in the digital context [7], while the second explores enabling new education and training strategies based on AI [104].”
Will Code Remain a Relevant User Interface for End-User Programming with Generative AI Models?
Sarkar, A. (2023) · Proceedings of the 2023 ACM SIGPLAN International Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software · doi:10.1145/3622758.3622882 · Cited by 23 · cited 1×
“Our vision exemplifies Sarkar’s hypothesis that Generative AI enables a ‘radical widening in scope and capability of end-user programming’ [82].”
A Hazard Analysis Framework for Code Synthesis Large Language Models
Khlaaf, H., Mishkin, P., Achiam, J., Krueger, G., & Brundage, M. (2022) · arXiv · doi:10.48550/arxiv.2207.14157 · Cited by 23
Enhancing Creativity in Large Language Models through Associative Thinking StrategiesPreprint
Mehrotra, P., Parab, A., & Gulwani, S. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2405.06715 · Cited by 19 · cited 2×
“For example, some research looks at boosting creativity within LLMs, such as Mehrotra et al. who examine whether associative thinking, which can help humans think creatively by linking seemingly unrelated concepts, is a useful prompting strategy to get LLMs to provide more creative outputs [107].”
Enforcing Resilience in Cyber-physical Systems via Equilibrium Verification at Runtime
Camilli, M., Mirandola, R., & Scandurra, P. (2023) · ACM Transactions on Autonomous and Adaptive Systems · doi:10.1145/3584364 · Cited by 16
Java Security: A Ten Year Retrospective
Gong, L. (2009) · Annual Computer Security Applications Conference · doi:10.1109/acsac.2009.44 · Cited by 11 · cited 2×
“Burnett concurs: ‘Considering software quality is necessary, because there is ample evidence that the programs end-users create are filled with expensive errors’.”
Self-sustaining Software Systems (S4): Towards Improved Interpretability and Adaptation
Cabrera, C., Paleyes, A., & Lawrence, N. D. (2024) · doi:10.1145/3643657.3643910 · Cited by 9 · cited 4×
“Addressing these limitations to offer sustainable software systems that self-maintain while keeping under end-users’ control is an important research area that requires our immediate focus [15].”
InfluentialSelf-citationtl;dr: Chill, y’all: AI Will Not Devour SE
Kang, E., & Shaw, M. (2024) · Proceedings of the 2024 ACM SIGPLAN International Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software · doi:10.1145/3689492.3689816 · Cited by 8 · cited 4×
“Instead of formal specifications that are ‘formal, static, and complete’, fitness of purpose can be checked with respect to ‘software credentials’, which are ‘heterogeneous, evolving, and incomplete’ [48].”
InfluentialIronies of Generative AI: Understanding and mitigating productivity loss in human-AI interactions
Simkute, A., Tankelevitch, L., Kewenig, V., Scott, A. E., Sellen, A., & Rintel, S. (2024) · arXiv · doi:10.48550/arxiv.2402.11364 · Cited by 7 · cited 1×
“Here are the requirements for an app to track the currency requirements 7 of an airline’s roster of pilots.”
Dataflow graphs as complete causal graphs
Paleyes, A., Guo, S., Schölkopf, B., & Lawrence, N. D. (2023) · IEEE/ACM International Conference on AI Engineering – Software Engineering for AI (CAIN) · doi:10.1109/cain58948.2023.00010 · Cited by 6 · cited 1×
“We can then treat this computational graph as a complete graphical causal model where practitioners can apply causal inference techniques to dataflow systems [73].”
Self-citationEnd-User Programming is WEIRD: How, Why and What to Do About It
Goel, H., Kumar, A., & Ragavan, S. S. (2023) · IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) · doi:10.1109/vl-hcc57772.2023.00013 · Cited by 5 · cited 1×
“The state of the art of research on autonomous deployment relies on continuous delivery, DevOps and MLOps tools [74].”
End-User Software Engineering Position Paper
Lieberman, H. A. (2007) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/dagsemproc.07081.15 · Cited by 2
A Reinforcement Learning-Based Service Model for the Internet of Things
Cabrera, C., & Clarke, S. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-91431-8_56 · Cited by 2 · cited 1×
“Hence, it may be easier or more effective to generate a textual explanation of the behaviour of a piece of code via its formal specification than to generate the text directly from the code.”
Self-citation
Research works printed but not linked: 31
- Jacob Austin Augustus Odena Maxwell Nye Maarten Bosma Henryk Michalewski David Dohan Ellen Jiang Carrie Cai Michael Terry Quoc Le et al. 2021. Program synthesis with large language models. arXiv:2108.07732. · arXiv
- Keith Bonawitz, Hubert Eichner, Wolfgang Grieskamp, Dzmitry Huba, Alex Ingerman, Vladimir Ivanov, Chloé Kiddon, Jakub Konečný, Stefano Mazzocchi, Brendan McMahan, et al. 2019. Towards federated learning at scale: System design. In Proceedings of the Machine Learning and Systems. A. Talwalkar, V. Smith, and M. Zaharia (Eds.), Vol. 1, 374–388. Retrieved from https://proceedings.mlsys.org/paper/2019/file/bd686fd640be98efaae0091fa301e613-Paper.pdf
- Christian Cabrera Andrei Paleyes Pierre Thodoroff and Neil D. Lawrence. 2023. Real-world machine learning systems: A survey from a data-oriented architecture perspective. arXiv:2302.04810. · arXiv
- Mark Chen Jerry Tworek Heewoo Jun Qiming Yuan Henrique Ponde de Oliveira Pinto Jared Kaplan Harri Edwards Yuri Burda Nicholas Joseph Greg Brockman et al. 2021. Evaluating large language models trained on code. arXiv:2107.03374 · arXiv
- Tyna Eloundou Sam Manning Pamela Mishkin and Daniel Rock. 2023. GPTs are GPTs: An early look at the labor market impact potential of large language models. arXiv:2303.10130. Retrieved from https://arxiv.org/abs/2303.10130 · arXiv
- Andrew D. Gordon Carina Negreanu José Cambronero Rasika Chakravarthy Ian Drosos Hao Fang Bhaskar Mitra Hannah Richardson Advait Sarkar Stephanie Simmons et al. 2023. Co-audit: Tools to help humans double-check AI-generated content. arXiv:2310.01297. · arXiv
- Phillip Guo. 2024. Beyond Just Programming: A Vision of Generative AI as an End-to End Expert Data Science Consultant. Cogna Tech Talk.
- Yanzhu Guo Guokan Shang Michalis Vazirgiannis and Chloé Clavel. 2023. The curious decline of linguistic diversity: Training language models on synthetic text. arXiv:2311.09807. · arXiv
- Sirui Hong Mingchen Zhuge Jonathan Chen Xiawu Zheng Yuheng Cheng Ceyao Zhang Jinlin Wang Zili Wang Steven Ka Shing Yau Zijuan Lin et al. 2023. MetaGPT: Meta programming for a multi-agent collaborative framework. arXiv:2308.00352 · arXiv
- Evan Hubinger Carson Denison Jesse Mu Mike Lambert Meg Tong Monte MacDiarmid Tamera Lanham Daniel M. Ziegler Tim Maxwell Newton Cheng et al. 2024. Sleeper agents: Training deceptive LLMs that persist through safety training. arXiv:2401.05566. Retrieved from https://arxiv.org/abs/2401.05566 · arXiv
- Carlos E. Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik R. Narasimhan. 2024. SWE-Bench: Can language models resolve real-world github issues?. In Proceedings of the 12th International Conference on Learning Representations. Retrieved from https://openreview.net/forum?id=VTF8yNQM66
- Haolin Jin Linghan Huang Haipeng Cai Jun Yan Bo Li and Huaming Chen. 2024. From LLMs to LLM-based agents for software engineering: A survey of current challenges and future. arXiv:2408.02479. Retrieved from https://arxiv.org/abs/2408.02479 · arXiv
- Neil D. Lawrence. 2017. Data readiness levels. arXiv:1705.02245. · arXiv
- K Rustan M Leino. 2010. Dafny: An automatic program verifier for functional correctness. In Proceedings of the International Conference on Logic for Programming Artificial Intelligence and Reasoning. Springer, 348–370.
- Zhenyan Lu Xiang Li Dongqi Cai Rongjie Yi Fangming Liu Xiwen Zhang Nicholas D. Lane and Mengwei Xu. 2024. Small language models: Survey measurements and insights. arXiv:2409.15790. · arXiv
- Shuming Ma Hongyu Wang Lingxiao Ma Lei Wang Wenhui Wang Shaohan Huang Li Dong Ruiping Wang Jilong Xue and Furu Wei. 2024. The era of 1-bit LLMs: All large language models are in 1.58 bits. arXiv:2402.17764. · arXiv
- Fangwen Mu Lin Shi Song Wang Zhuohao Yu Binquan Zhang Chenxue Wang Shichao Liu and Qing Wang. 2023. ClarifyGPT: Empowering LLM-based code generation with intention clarification. arXiv:2310.10996. · arXiv
- Emerson Murphy-Hill, Alberto Elizondo, Ambar Murillo, Marian Harbach, Bogdan Vasilescu, Delphine Carlson, and Florian Dessloch. 2024. GenderMag improves discoverability in the field, especially for women. In Proceedings of the 2024 IEEE/ACM 46th International Conference on Software Engineering (ICSE ’24). IEEE Computer Society, 973–973.
- Cyrus Omar, Patrick Ferris, and Anil Madhavapeddy. 2024. Modularizing reasoning about AI capabilities via abstract dijkstra monads. In Proceedings of the 12th ACM SIGPLAN Workshop on Higher-Order Programming with Effects (HOPE’24).
- OpenAI. 2024. GPT-4 technical report. arXiv:2303.08774. · arXiv
- 10.5555/287893.287899 · doi:10.5555/287893.287899
- Martin Porcheron, Joel E. Fischer, and Stuart Reeves. 2021. Pulling back the curtain on the wizards of Oz. Proceedings of the ACM on Human-Computer Interaction 4, CSCW3 (Jan. 2021), 1–22.
- Leo Porter and Daniel Zingaro. 2023. Learn AI-Assisted Python Programming with GitHub Copilot and ChatGPT. Manning Publications, New York, NY.
- Bogdana Rakova, Jingying Yang, Henriette Cramer, and Rumman Chowdhury. 2021. Where responsible AI meets reality: Practitioner perspectives on enablers for shifting organizational practices. Proceedings of the ACM on Human-Computer Interaction 5, CSCW1 (2021), 7:1–7:23.
- Mary Shaw. 2022. Myths and mythconceptions: What does it mean to be a programming language, anyhow? Proceedings of the ACM on Programming Languages 4, HOPL (2022), 1–44.
- Ilia Shumailov Zakhar Shumaylov Yiren Zhao Yarin Gal Nicolas Papernot and Ross Anderson. 2023. The curse of recursion: Training on generated data makes models forget. arXiv:2305.17493. · arXiv
- Apoorva Srivastava, Sukriti Bhardwaj, and Shipra Saraswat. 2017. SCRUM model for agile methodology. In Proceedings of the 2017 International Conference on Computing, Communication and Automation (ICCCA ’17). IEEE, 864–869.
- Eric Thorn Shawn C. Kimmel and Michelle Chaka. 2018. A Framework for Automated Driving System Testable Cases and Scenarios. Technical Report DOT HS 812 623. United States Department of Transportation. Retrieved from https://doi.org/10.48550/arXiv.2312.11805 · DOI · Paper page
- Michele Tufano Anisha Agarwal Jinu Jang Roshanak Zilouchian Moghaddam and Neel Sundaresan. 2024. AutoDev: Automated AI-driven development. arXiv:2403.08299. · arXiv
- Changjie Wang Mariano Scazzariello Alireza Farshin Dejan Kostic and Marco Chiesa. 2023. Making network configuration human friendly. arxiv:2309.06342. · arXiv
- Qingyun Wu Gagan Bansal Jieyu Zhang Yiran Wu Beibin Li Erkang Zhu Li Jiang Xiaoyun Zhang Shaokun Zhang Jiale Liu et al. 2023. AutoGen: Enabling next-gen LLM applications via multi-agent conversation. arXiv:2308.08155. · arXiv
Cited by
Bandi, A., Kongari, B., Naguru, R., Pasnoor, S., & Vilipala, S. V. (2025) · Future Internet · doi:10.3390/fi17090404 · Cited by 207
Semantic Commit: Helping Users Update Intent Specifications for AI Memory at Scale
Vaithilingam, P., Kim, M. G., Acosta-Parenteau, F.-C., Lee, D. W., Mhedhbi, A., Glassman, E. L., & Arawjo, I. (2025) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3746059.3747778 · Cited by 31
Machine Learning Systems: A Survey from a Data-Oriented Perspective
Cabrera, C., Paleyes, A., Thodoroff, P., & Lawrence, N. D. (2025) · ACM Computing Surveys · doi:10.1145/3769292 · Cited by 19
Self-citationA 2030 Roadmap for Software Engineering
Pezzè, M., Abrahão, S., Penzenstadler, B., Poshyvanyk, D., Roychoudhury, A., & Yue, T. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3731559 · Cited by 13
User stories as boundary objects in agile requirements engineering: A theoretical literature review
Sporsem, T., Dingsøyr, T., & Stol, K.-J. (2025) · Journal of Systems and Software · doi:10.1016/j.jss.2025.112693 · Cited by 16
Metadata
- Authors
- Robinson, D., Cabrera, C., Gordon, A. D., Lawrence, N. D., & Mennen, L.
- Affiliations
- University of Cambridge (GB)
- University of Edinburgh (GB)
- Cogna, London, United Kingdom of Great Britain and Northern Ireland
- Venue
- ACM Transactions on Software Engineering and Methodology
- Published
- 2025-05-26
- Volume
- 34
- Issue
- 5
- Pages
- 1-22
- ISSN
- 1049-331X (ISSN-L)
- 1557-7392
- OpenAlex
- W4405396181
- Cited by
- 12
- References
- 67
- Citations by year
- 2026: 5 · 2025: 6
- FWCI
- 4.53
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- ACM Transactions on Software Engineering and Methodology (publishedVersion, cc-by)
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
- Physical Sciences › Computer Science › Software › Spreadsheets and End-User Computing
- Physical Sciences › Computer Science › Software › Advanced Software Engineering Methodologies
- Keywords
- end-user software engineering, requirements engineering, generative AI, natural language requirements, software development lifecycle, software generation, software maintenance
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.