The DOI matches its Crossref record; 64 of the 64 references it cites were found in a registry or an index.
ChatGPT, Copilot, Gemini, SciSpace and Wolfram versus higher education assessments: an updated multi-institutional study of the academic integrity impacts of Generative Artificial Intelligence (GenAI) on assessment, teaching and learning in engineering
Open accessAbstract
More than a year has passed since reports of ChatGPT-3.5’s capability to pass exams sent shockwaves through education circles. These initial concerns led to a multi-institutional and multi-disciplinary study to assess the performance of Generative Artificial Intelligence (GenAI) against assessment tasks used across 10 engineering subjects, showcasing the capability of GenAI. Assessment types included online quiz, numerical, oral, visual, programming and writing (experimentation, project, reflection and critical thinking, and research). Twelve months later, the study was repeated using new and updated tools ChatGPT-4, Copilot, Gemini, SciSpace and Wolfram. The updated study investigated the performance and capability differences, identifying the best tool for each assessment type. The findings show that increased performance and features can only heighten academic integrity concerns. While cheating concerns are central, opportunities to integrate GenAI to enhance teaching and learning are possible. While each GenAI tool had specific strengths and weaknesses, ChatGPT-4 was well-rounded. A GenAI Assessment Security and Opportunity Matrix is presented to provide the community practical guidance on managing assessment integrity risks and integration opportunities to enhance learning.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“this tendency to not disclose the use of GenAI is becoming an increasing problem within research and extends beyond students (Nikolic et al., 2024).”
From A systematic literature review of attitudes, intentions and behaviours of teaching academics pertaining to AI and generative AI (GenAI) in higher education: An analysis of GenAI adoption using the UTAUT framework (2024) · cites this paper as Nikolic et al., 2024
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
2 retracted
Zhang, M., Wu, L., Yang, T., Zhu, B., & Liu, Y. (2024) · Surfaces and Interfaces · doi:10.1016/j.surfin.2024.104081 · Cited by 39 · cited 1×
“At the time of writing, two examples of such papers with the accidental prompting visible and no ethics declaration of GenAI use include Zhang et al. (2024) and Bader et al. (2024).”
RetractedBader, R., Imam, A., Alnees, M., Adler, N., ilia, J., Zugayar, D., Dan, A., & Khalaileh, A. (2024) · Radiology Case Reports · doi:10.1016/j.radcr.2024.02.037 · Cited by 14 · cited 1×
“At the time of writing, two examples of such papers with the accidental prompting visible and no ethics declaration of GenAI use include Zhang et al. (2024) and Bader et al. (2024).”
RetractedChatting and cheating: Ensuring academic integrity in the era of ChatGPT
Cotton, D. R. E., Cotton, P. A., & Shipway, J. R. (2024) · Innovations in Education and Teaching International · doi:10.1080/14703297.2023.2190148 · Cited by 3,087 · cited 1×
“One key risk, front and centre in the minds of academics across the world, is that associated with academic integrity (Cotton, Cotton, and Shipway 2023).”
Chatting and Cheating. Ensuring academic integrity in the era of ChatGPT
Cotton, D., Cotton, P., & Shipway, J. R. (2023) · doi:10.35542/osf.io/mrz8h · Cited by 3,079
An Overview of Chatbot Technology
Adamopoulou, E., & Moussiades, L. (2020) · IFIP Advances in Information and Communication Technology · doi:10.1007/978-3-030-49186-4_31 · Cited by 990 · cited 2×
“The next major evolution came with the integration of Natural Language Processing (NLP) and virtual personal assistants such as Siri and Alexa, which made these technologies more user-friendly and wide-spread by making the conversations feel more life-like (Adamopoulou and Moussiades 2020).”
A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT
White, J., Fu, Q., Hays, S., Sandborn, M., Olea, C., Gilbert, H., Elnashar, A., Spencer-Smith, J., & Schmidt, D. C. (2023) · arXiv · doi:10.48550/arxiv.2302.11382 · Cited by 838
Bahroun, Z., Anane, C., Ahmed, V., & Zacca, A. (2023) · Sustainability · doi:10.3390/su151712983 · Cited by 784 · cited 2×
“The release of ChatGPT 3 to the general public in November 2022 sent shockwaves through education institutions when its capability to disrupt traditional academic integrity safeguards and transform teaching and learning was realised (Bahroun et al. 2023).”
(2023) · Journal of Applied Learning & Teaching · doi:10.37074/jalt.2023.6.1.23 · Cited by 421
A large-scale comparison of human-written versus ChatGPT-generated essays
Herbold, S., Hautli-Janisz, A., Heuer, U., Kikteva, Z., & Trautsch, A. (2023) · Scientific Reports · doi:10.1038/s41598-023-45644-9 · Cited by 338 · cited 1×
“For example, GenAI can generate essays rated higher quality than those produced by humans (Herbold et al. 2023); pass questions involving communication skills, ethics, empathy, and pro-fessionalism in a United States Medical Licensing Examination (Brin et al. 2023); pass board-style questions…”
Nikolic, S., Daniel, S., Haque, R., Belkina, M., Hassan, G. M., Grundy, S., Lyden, S., Neal, P., & Sandison, C. (2023) · European Journal of Engineering Education · doi:10.1080/03043797.2023.2213169 · Cited by 336 · cited 2×
“This paper also includes GPTs (Wolfram for numerical-based assessments and SciSpace for research-based assessments) in ChatGPT-4 to check against the performance predictions made in Nikolic et al. (2023a).”
Self-citation
Research works printed but not linked: 6
- Borsotti, V., A. Begel, and P. Bjorn. 2024. Neurodiversity and the Accessible University: Exploring Organizational Barriers.” In Proceedings of the ACM on Human-Computer Interaction. Vol. 8. Access Labor and Opportunities for Change.
- Chung, H.-H., F.-L. Chung, S.-M. Lin, and Y.-J. Lan. 2023. “Tools and Approaches of Generative Artificial Intelligence Used in Education.” 31st International Conference on Computers in Education, Japan.
- Daniel, S., and S. Nikolic. 2023. “Benchmarking AI Tools and Assessing Integrity: Assessment Integrity in the AI Age.” In SEFI Ethics SIG. Editorial. https://www.sefi.be/2023/10/14/benchmarking-ai-tools-and-assessing-integrity-assessment-integrity-in-the-ai-age/.
- Groch, S. 2024. “These Students Cheated on a Test and Got Away with It. Then the Blackmail Started.” Sydney Morning Herald. Accessed March 29, 2024. https://www.smh.com.au/education/these-students-cheated-on-a-test-and-got-away-with-it-then-the-blackmail-started-20240320-p5fe1x.html?btis=
- 10.1080/03043797.2023.2213169 · DOI · Paper page
- Purtill, J. 2023. “ChatGPT Was Tipped to Cause Widespread Cheating. Here’s What Students Say Happened.” ABC Australia. Accessed November 22, 2023. https://www.abc.net.au/news/science/2023-11-22/how-high-school-students-used-chatgpt-2023-education-cheating/103108620.
Cited by
Implementing generative AI (GenAI) in higher education: A systematic review of case studies
Belkina, M., Daniel, S., Nikolic, S., Haque, R., Lyden, S., Neal, P., Grundy, S., & Hassan, G. M. (2025) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2025.100407 · Cited by 183
Self-citationNikolic, S., Wentworth, I., Sheridan, L., Moss, S., Duursma, E., Jones, R. A., Ros, M., & Middleton, R. (2024) · Australasian Journal of Educational Technology · doi:10.14742/ajet.9643 · Cited by 102
“Other prominent large language models include Google’s Gemini, Meta’s LLaMA and Anthropic's Claude (Nikolic et al., 2024).”
InfluentialSelf-citationTran, M., Balasooriya, C. D., Jonnagaddala, J., Leung, G., Mahboobani, N. R., Ramani, S., Rhee, J., Schuwirth, L., Najafzadeh-Tabrizi, N. S., Semmler, C., et al. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01721-z · Cited by 71
Deroncele-Acosta, Á., Sayán-Rivera, R. M. E., López, Á. D. M., & Norabuena-Figueroa, E. (2025) · Applied System Innovation · doi:10.3390/asi8030083 · Cited by 77
Humble, N. (2024) · International Journal of Educational Technology in Higher Education · doi:10.1186/s41239-024-00494-x · Cited by 55
“…et al., 2024; Liao et al., 2024; Liffiton et al., 2023; Lin et al., 2024; López-Pernas et al., 2023; Lyu et al., 2024; MacNeil et al., 2024; Nikolic et al., 2024; Nizamudeen et al., 2024; Ouhbi, 2024; Pankiewicz & Baker, 2024; Phung et al., 2024; Prather et al., 2023; Sheese et al., 2024;…”
Metadata
- Authors
- Nikolic, S., Sandison, C., Haque, R., Daniel, S., Grundy, S., Belkina, M., Lyden, S., Hassan, G. M., & Neal, P.
- Affiliations
- University of Wollongong (AU)
- University of the Sunshine Coast (AU)
- University of Technology Sydney (AU)
- UNSW Sydney (AU)
- Western Sydney University (AU)
- University of Tasmania (AU)
- The University of Western Australia (AU)
- Corresponding author
- Nikolic, Sasha
- ORCID
- Venue
- Australasian Journal of Engineering Education
- Publisher
- Informa UK Limited
- Published
- 2024-07-02
- Volume
- 29
- Issue
- 2
- Pages
- 126-153
- ISSN
- 1324-5821 (ISSN-L)
- 2205-4952
- 1325-4340
- OpenAlex
- W4400530818
- Cited by
- 98
- References
- 61
- Citations by year
- 2026: 36 · 2025: 53 · 2024: 9
- FWCI
- 40.74
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Australasian journal of engineering education (publishedVersion, cc-by-nc-nd)
- OPUS - Open Publications of UTS Scholars (University of Technology Sydney) (submittedVersion, other-oa)
- Figshare (submittedVersion, cc-by)
- UWA Profiles and Research Repository (UWA) (submittedVersion, cc-by-nc-nd)
- UNSWorks (UNSW Sydney) (submittedVersion, cc-by-nc-nd)
- Figshare (cc-by)
- Funders
- Australian Council of Engineering Deans (ACED) and the Australasian Association of Engineering Education
- Topics
- Social Sciences › Education › Academic integrity and plagiarism
- Social Sciences › Education › Artificial Intelligence in Education
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Keywords
- generative AI, academic integrity, engineering education, assessment, ChatGPT-4, Copilot, Gemini, SciSpace, Wolfram Alpha, cheating, AI-assisted learning, assessment security
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: ChatGPT, Copilot, Gemini, SciSpace and Wolfram versus higher education assessments: an updated multi-institutional study of the academic integrity impacts of Generative Artificial Intelligence (GenAI) on assessment, teaching and learning in engineering. https://app.cochat.ai/papers/10.1080/22054952.2024.2372154
- The paper
- Nikolic, S., Sandison, C., Haque, R., Daniel, S., Grundy, S., Belkina, M., Lyden, S., Hassan, G. M., & Neal, P. (2024). ChatGPT, Copilot, Gemini, SciSpace and Wolfram versus higher education assessments: an updated multi-institutional study of the academic integrity impacts of Generative Artificial Intelligence (GenAI) on assessment, teaching and learning in engineering. Australasian Journal of Engineering Education. https://doi.org/10.1080/22054952.2024.2372154
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.