Exploring the Integration of Generative AI Tools in Software Testing Education: A Case Study on ChatGPT and Copilot for Preparatory Testing Artifacts in Postgraduate Learning
Open accessAbstract
Software testing education is important for building qualified testing professionals. To ensure that software testing graduates are ready for real-world challenges, it is necessary to integrate modern tools and technologies into the curriculum. With the emergence of Large Language Models (LLMs), their potential use in software engineering has become a focus, but their application in software testing education remains largely unexplored. This study, conducted in the Capstone Project course of a postgraduate software testing program, was carried out over two semesters with two distinct groups of students. A custom-built Travel Application limited to a web platform was used in the first semester. In the second semester, a new set of students worked with an open-source application, offering a larger-scale, multi-platform experience across web, desktop, and mobile platforms. Students initially created preparatory testing artifacts manually as a group deliverable. Following this, they were assigned an individual assignment to generate the same artifacts using LLM tools such as ChatGPT 3.5 in the first semester and Microsoft Copilot in the second. This process directly compared manually created artifacts and those generated using LLMs, leveraging AI for faster outputs. After completion, they responded to a set of assigned questions. The students’ responses were assessed using an integrated methodology, including quantitative and qualitative assessments, sentiment analysis to underst…
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Generative AI tools such as ChatGPT, GitHub Copilot, and Gemini have become embedded in the daily workflow of graduate students.
“Generative Artificial Intelligence (AI) tools such as ChatGPT, GitHub Copilot, and Gemini have become embedded in the daily workflow of graduate students [12, 17, 19].”
From Students' Practices and Skills in the LLM-Era: "You Can't Outsource the Struggle and Still Get the Skill" (2026) · cites this paper as [12]
Cites
VADER: a parsimonious rule-based model for sentiment analysis of social media text
Hutto, C., & Gilbert, E. (2014) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v8i1.14550 · Cited by 6,087 · see citing works
Prompt Engineering with ChatGPT: A Guide for Academic Writers
Giray, L. (2023) · Annals of Biomedical Engineering · doi:10.1007/s10439-023-03272-4 · Cited by 776 · see citing works
Ensemble of keyword extraction methods and classifiers in text classification
Onan, A., Korukoğlu, S., & Bulut, H. (2016) · Expert Systems with Applications · doi:10.1016/j.eswa.2016.03.045 · Cited by 630 · see citing works
Software Testing With Large Language Models: Survey, Landscape, and Vision
Wang, J., Huang, Y., Chen, C., Liu, Z., Wang, S., & Wang, Q. (2024) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2024.3368208 · Cited by 573 · see citing works
Exploring Students’ Perceptions of ChatGPT: Thematic Analysis and Follow-Up Survey
Shoufan, A. (2023) · IEEE Access · doi:10.1109/access.2023.3268224 · Cited by 499 · cited 1× · see citing works
“Recent studies, such as Shoufan [39], utilized thematic analysis to explore students’ perceptions of ChatGPT, cate-gorizing their responses into recurring themes.”
An Empirical Evaluation of Using Large Language Models for Automated Unit Test Generation
Schäfer, M., Nadi, S., Eghbali, A., & Tip, F. (2024) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2023.3334955 · Cited by 431 · cited 1× · see citing works
“In the software testing paradigm, the application of LLM tools has been widely studied in various activities [19], [20] including unit test generation [21], [22], regression test case generation [23], test script generation [22], and impact analysis of requirement changes [24].”
Prompt Engineering in Large Language Models
Marvin, G., Hellen, N., Jjingo, D., & Nakatumba-Nabende, J. (2024) · Algorithms for Intelligent Systems · doi:10.1007/978-981-99-7962-2_30 · Cited by 403 · see citing works
Computer Science Curricula 2023
Kumar, A. N., Raj, R. K., Aly, S. G., Anderson, M., Becker, B. A., Blumenthal, R. L., Eaton, E., Epstein, S. L., Goldweber, M., Jalote, P., et al. (2024) · ACM eBooks · doi:10.1145/3664191 · Cited by 235 · see citing works
Code-Aware Prompting: A Study of Coverage-Guided Test Generation in Regression Setting using LLM
Ryan, G., Jain, S., Shang, M., Wang, S., Ma, X., Ramanathan, M. K., & Ray, B. (2024) · Proceedings of the ACM on software engineering. · doi:10.1145/3643769 · Cited by 125 · see citing works
Software-testing education: A systematic literature mapping
Garousi, V., Rainer, A., Lauvås, P., & Arcuri, A. (2020) · Journal of Systems and Software · doi:10.1016/j.jss.2020.110570 · Cited by 121 · see citing works
Buenaño-Fernández, D., González, M., Gil, D., & Luján‐Mora, S. (2020) · IEEE Access · doi:10.1109/access.2020.2974983 · Cited by 115 · see citing works
ChatGPT vs SBST: A Comparative Assessment of Unit Test Suite Generation
Tang, Y., Liu, Z., Zhou, Z., & Luo, X. (2024) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2024.3382365 · Cited by 109 · see citing works
Automatic Generation of Acceptance Test Cases From Use Case Specifications: An NLP-Based Approach
Wang, C., Pastore, F., Göknil, A., & Briand, L. (2020) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2020.2998503 · Cited by 103 · see citing works
Using sentiment analysis to evaluate qualitative students’ responses
Dake, D. K., & Gyimah, E. (2022) · Education and Information Technologies · doi:10.1007/s10639-022-11349-1 · Cited by 91 · see citing works
Linking Requirements and Testing in Practice
Uusitalo, E., Komssi, M., Kauppinen, M., & Davis, A. M. (2008) · IEEE International Requirements Engineering Conference (RE) · doi:10.1109/re.2008.30 · Cited by 78 · see citing works
Software Engineering Education Must Adapt and Evolve for an LLM Environment
Kirova, V. D., Ku, C. S., Laracy, J. R., & Marlowe, T. J. (2024) · ACM Technical Symposium on Computer Science Education (SIGCSE) · doi:10.1145/3626252.3630927 · Cited by 70 · see citing works
Large Language Models in Education: A Systematic Review
Dong, B., Bai, J., Xu, T., & Zhou, Y. (2024) · International Conference on Computer Science and Technologies in Education (CSTE) · doi:10.1109/cste62025.2024.00031 · Cited by 63 · see citing works
Sentiment analysis for formative assessment in higher education: a systematic literature review
Grimalt‐Álvaro, C., & Usart, M. (2023) · Journal of Computing in Higher Education · doi:10.1007/s12528-023-09370-5 · Cited by 55 · see citing works
ChatGPT for Learning HCI Techniques: A Case Study on Interviews for Personas
Barambones, J., Moral, C., de Antonio, A., Imbert, R., Normand, L. M., & Villalba‐Mora, E. (2024) · IEEE Transactions on Learning Technologies · doi:10.1109/tlt.2024.3386095 · Cited by 47 · see citing works
Santos, R., Santos, Í., de Magalhães, C. V. C., & Santos, R. D. S. (2024) · IEEE International Conference on Software Testing, Verification and Validation (ICST) · doi:10.1109/icst60714.2024.00039 · Cited by 40 · see citing works
Pirzado, F. A., Ahmed, A., Mendoza-Urdiales, R. A., & Terashima‐Marín, H. (2024) · IEEE Access · doi:10.1109/access.2024.3443621 · Cited by 37 · see citing works
Palaute: An Online Text Mining Tool for Analyzing Written Student Course Feedback
Grönberg, N., Knutas, A., Hynninen, T., & Hujala, M. (2021) · IEEE Access · doi:10.1109/access.2021.3116425 · Cited by 33 · see citing works
Lang, Q., Tian, S., Wang, M., & Wang, J. (2024) · IEEE Transactions on Learning Technologies · doi:10.1109/tlt.2024.3456128 · Cited by 25 · see citing works
ChatGPT’s Impact on Education and Healthcare: Insights, Challenges, and Ethical Consideration
Hamad, Z. T., Jamil, N., & Belkacem, A. N. (2024) · IEEE Access · doi:10.1109/access.2024.3437374 · Cited by 25 · see citing works
Software Testing Research Challenges: An Industrial Perspective
Alshahwan, N., Harman, M., & Marginean, A. (2023) · IEEE International Conference on Software Testing, Verification and Validation (ICST) · doi:10.1109/icst57152.2023.00008 · Cited by 21 · see citing works
AI-Powered Software Testing: The Impact of Large Language Models on Testing Methodologies
Bayrı, V., & Demirel, E. (2023) · International Informatics and Software Engineering Conference (IISEC) · doi:10.1109/iisec59749.2023.10391027 · Cited by 20 · see citing works
Effective Lesson Planning and Assessment Design Using Leveraging Microsoft Copilot Implementation
Ramos, R. F., Angel, R. M. D., Ruetas, A. P., Lagman, A. C., Morano, J., Payongayong, R. T., & Enrile, J. C. (2024) · IEEE Control and System Graduate Research Colloquium (ICSGRC) · doi:10.1109/icsgrc62081.2024.10691289 · Cited by 19 · see citing works
Utilizing open source software in teaching practice-based software engineering courses
Dorodchi, M., & Dehbozorgi, N. (2016) · IEEE Frontiers in Education Conference (FIE) · doi:10.1109/fie.2016.7757683 · Cited by 19 · see citing works
Teaching Software Testing with Free and Open Source Software
Deng, L., Dehlinger, J., & Chakraborty, S. (2020) · IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW) · doi:10.1109/icstw50294.2020.00074 · Cited by 16 · see citing works
Naimi, L., Bouziane, E. M., Manaouch, M., & Jakimi, A. (2024) · International Conference on Circuit, Systems and Communication (ICCSC) · doi:10.1109/iccsc62074.2024.10616548 · Cited by 14 · see citing works
Arlitt, M. F., Coughlin, T., Faraboschi, P., Frachtenberg, E., Laplante, P., Milojicic, D. S., Patel, N., & Saracco, R. (2023) · Computer · doi:10.1109/mc.2022.3203505 · Cited by 14 · see citing works
Rahman, T., Nwokeji, J. C., Matovu, R., Frezza, S. T., Sugnanam, H., & Pisolkar, A. (2021) · IEEE Frontiers in Education Conference (FIE) · doi:10.1109/fie49875.2021.9637220 · Cited by 12 · see citing works
Analyzing Software Industry Trends to Improve Curriculum
Siddique, A., Butt, G. M., Zahid, A., Naveed, Q. N., & Alouane, M. T.-H. (2024) · IEEE Access · doi:10.1109/access.2024.3362244 · Cited by 12 · see citing works
Roadmap for Software Engineering Education using ChatGPT
Abdelfattah, A. M., Ali, N. A., Elaziz, M. A., & Ammar, H. H. (2023) · International Conference on Artificial Intelligence Science and Applications in Industry and Society (CAISAIS) · doi:10.1109/caisais59399.2023.10270477 · Cited by 11 · see citing works
Innovative Approaches to Impact Analysis of Requirement Changes using LLM in Software Projects
Rusdianto, D. S., Fabroyir, H., & Yuhana, U. L. (2024) · IEEE International Symposium on Consumer Electronics (ISCE) · doi:10.1109/isct62336.2024.10791169 · Cited by 8 · see citing works
Comparative Analysis of Generative AI Tools in Enhancing Educational Engagement
Borović, F., Aleksić-Maslać, K., & Vranešić, P. (2024) · International Convention on Information and Communication Technology, Electronics and Microelectronics · doi:10.1109/mipro60963.2024.10569483 · Cited by 6 · see citing works
Teaching Software Testing to Industrial Practitioners Using Distance and Web-Based Learning
Enoiu, E. P. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-57663-9_6 · Cited by 6 · see citing works
Industry perceptions of the competencies needed by novice software tester
Hamid, B., & Ikram, N. (2023) · Education and Information Technologies : Official Journal of the IFIP technical committee on Education · doi:10.1007/s10639-023-12055-2 · Cited by 5 · see citing works
WIP: Assessing the Effectiveness of ChatGPT in Preparatory Testing Activities
Haldar, S., Pierce, M., & Capretz, L. F. (2024) · IEEE Frontiers in Education Conference (FIE) · doi:10.1109/fie61694.2024.10893214 · Cited by 4 · see citing works
Self-citationAludhilu, H., & Sutinen, E. (2024) · ACM/IEEE International Workshop on Software-intensive Business (IWSiB) · doi:10.1145/3643690.3648244 · Cited by 4 · see citing works
Importance of RTM for Testing a Web-based Project
Madan, M., Dave, M., & Tandon, A. (2018) · International Conference on Reliability, Infocom Technologies and Optimization (Trends and Future Directions) (ICRITO) · doi:10.1109/icrito.2018.8748299 · Cited by 4 · see citing works
Bilow, F., & DeWaters, J. (2023) · IEEE Frontiers in Education Conference (FIE) · doi:10.1109/fie58773.2023.10342901 · Cited by 3 · see citing works
Implementation of Automatic Keyword Extraction Software for English Text Based on Python
Liang, X., He, J., & Sun, Q. (2022) · International Conference on Applied Machine Learning (ICAML) · doi:10.1109/icaml57167.2022.00031 · Cited by 2 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Generative AI in Agile Software Development: A Comprehensive Survey
Malladi, N. V., & Reddy, K. S. (2025) · International Conference on Innovative Data Communication Technologies and Application (ICIDCA) · doi:10.1109/icidca66325.2025.11280486 · Cited by 21 · see citing works
Wang, Z. (2025) · Journal of Computer and Communications · doi:10.4236/jcc.2025.138010 · Cited by 1 · see citing works
On the Use of AI in Signal Processing Programming Applications at the Master of Science Level
Druga, C. G., Sîrbu, G., & Aiordăchioaie, D. (2025) · doi:10.1109/iseee67817.2025.11304834 · Cited by 1 · see citing works
de Freitas, E. R. S., Pinto, G., & Monteiro, D. (2026) · arXiv · doi:10.48550/arxiv.2607.29519
“Generative Artificial Intelligence (AI) tools such as ChatGPT, GitHub Copilot, and Gemini have become embedded in the daily workflow of graduate students [12, 17, 19].”
Exploring the Use of Generative AI in Software Development: A Preliminary Study
Mar’i, F., Nerisafitra, P., Marianingsih, S., Hanjaya, S., Aisyah, Y., Primatama, D., & Sutaji, D. (2025) · E3S Web of Conferences · doi:10.1051/e3sconf/202564504002
Um chatbot treinado para entender intenções de estudantes em diálogos sobre teste de software
Musico, T., Melo, S. M., Paschoal, L. N., & Souza, S. R. S. (2025) · Anais do ... Simpósio Brasileiro de Informática na Educação/Anais do Simpósio Brasileiro de Informática na Educação · doi:10.5753/sbie.2025.12704
Comparative Analysis of Generative AI Performance in University Programming Courses
Suomalainen, E., Vanhala, E., & Ikonen, J. (2026) · Lecture notes in business information processing · doi:10.1007/978-3-032-14518-5_16
Dohare, S., Chouragade, G., & Bangade, A. (2026) · International Conference on Machine Learning and Autonomous Systems (ICMLAS) · doi:10.1109/icmlas67792.2026.11483906
Akter, M., Spriha, A. B., Islam, S., Jahan, N., Rahman, N. Z., Arman, S. E., & Jamal, L. (2026) · International Conference on Information and Education Technology (ICIET) · doi:10.1109/iciet69664.2026.11561611
How students use generative AI for software testing: An observational study
Ardıç, B., Dilavrec, Q. L., & Zaidman, A. (2026) · Empirical Software Engineering · doi:10.1007/s10664-026-10898-0 · Cited by 2 · see citing works
Software Testing Education in the LLM Era: Insights and Emerging Theory
Dwarakanath, S., do Nascimento, N. M., & Guimarães, E. T. (2026) · doi:10.1145/3793654.3793757
The 3D4E strategic approach: Can GAI improve visual thinking skills in STEM high schools?
Abouelenein, Y. A. M., Selim, S. A. S., Azim, Z. M. A., & Khalaf, M. H. R. (2026) · Education and Information Technologies · doi:10.1007/s10639-026-14093-y
Álvarez-Jiménez, M., Pomagualli, A. G. C., Gamboa, D., & Oscullo, D. S. P. (2026) · Computers · doi:10.3390/computers15070466
Lima, J. B., & Lima, M. S. (2026) · Anais do Workshop sobre Educação em Computação (WEI) · doi:10.5753/wei.2026.21084
O Uso de IA Generativa no Ensino de Projeto de Software
Sabbatini, L., Motta, R., & Neves, V. D. O. (2026) · Anais do Workshop sobre Educação em Computação (WEI) · doi:10.5753/wei.2026.22731
AI-Driven Teaching Reform in Software Engineering: A Focus on Prototype Design
Wu, Y., Xu, L., Sun, X.-Y., Chen, E.-H., & Pan, H.-Q. (2025) · International Symposium on Computational Intelligence and Design · doi:10.1109/iscid68789.2025.00067
Ali, H., Cao, Z., & Gao, F. (2026) · Journal of the Franklin Institute · doi:10.1016/j.jfranklin.2026.109173
- Authors
- Haldar, S., Pierce, M., & Fernando Capretz, L.
- Affiliations
- Fanshawe College (CA)
- Department of Electrical and Computer Engineering, Western University, London, ON, Canada
- Venue
- IEEE Access
- Published
- 2025
- Volume
- 13
- Pages
- 46070-46090
- ISSN
- 2169-3536 (ISSN-L)
- OpenAlex
- W4407948501
- Cited by
- 16
- References
- 43
- Citations by year
- 2026: 10 · 2025: 6
- FWCI
- 16.77
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- IEEE Access (publishedVersion, cc-by)
- Scholarship@Western (Western University) (submittedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Physical Sciences › Computer Science › Information Systems › Artificial Intelligence in Education
- Physical Sciences › Computer Science › Computer Science Applications › Teaching and Learning Programming
- Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
- Keywords
- generative AI in education, software testing education, large language models, ChatGPT, Microsoft Copilot, manual testing, AI-assisted learning, travel applications
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
