Does ChatGPT Play a Double-Edged Sword Role in the Field of Higher Education? An In-Depth Exploration of the Factors Affecting Student Performance
Open accessThe DOI matches its Crossref record; 37 of the 37 references it cites were found in a registry or an index.
Abstract
The application of generative artificial intelligence in the field of education has been receiving increasing attention, with the performance of chatbot ChatGPT being particularly prominent. This study aims to explore in depth the performance impact on higher education students utilizing ChatGPT. To this end, we conducted a survey on 448 university students and employed the partial-least squares (PLS) method of structural equation modeling for data analysis. The results indicate that all eight hypothetical paths posited in this study were supported, and surprisingly, the hypothesis that technology characteristics have a direct effect on performance impact was supported. Moreover, the study found that overall quality is a crucial factor determining performance impact. Overall quality indirectly affects performance impact through task-technology fit, technology characteristics, and compatibility, among which the mediating effect of compatibility is most significant, followed by technology characteristics. This study offers practical recommendations for students on the proper use of ChatGPT during the learning process and assists developers in enhancing the services of the ChatGPT system.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ChatGPT for Good? On Opportunities and Challenges of Large Language Models for Education
Kasneci, E., Seßler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., et al. (2023) · doi:10.35542/osf.io/5er8f · Cited by 7,552
ChatGPT for good? On opportunities and challenges of large language models for education
Kasneci, E., Sessler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., et al. (2023) · Learning and Individual Differences · doi:10.1016/j.lindif.2023.102274 · Cited by 7,326
Knowledge management: An organizational capabilities perspective
Gold, A. H., Malhotra, A., & Segars, A. H. (2001) · Journal of Management Information Systems · doi:10.1080/07421222.2001.11045669 · Cited by 6,233
Task-Technology Fit and Individual Performance
Goodhue, D. L., & Thompson, R. L. (1995) · MIS Quarterly · doi:10.2307/249689 · Cited by 6,006
Dwivedi, Y. K., Hughes, L., Ismagilova, E., Aarts, G., Coombs, C., Crick, T., Duan, Y., Dwivedi, R., Edwards, J., Eirug, A., et al. (2021) · International Journal of Information Management · doi:10.1016/j.ijinfomgt.2019.08.002 · Cited by 4,611
Sallam, M. (2023) · Healthcare · doi:10.3390/healthcare11060887 · Cited by 3,126
Baidoo-Anu, D., & Owusu Ansah, L. (2023) · doi:10.2139/ssrn.4337484 · Cited by 3,004
Wu, J.-H., & Wang, S.-C. (2005) · Information & Management · doi:10.1016/j.im.2004.07.001 · Cited by 2,284
Gilson, A., Safranek, C. W., Huang, T., Socrates, V., Chi, L., Taylor, R. A., & Chartash, D. (2023) · JMIR Medical Education · doi:10.2196/45312 · Cited by 2,078
What if the devil is my guardian angel: ChatGPT as a case study of using chatbots in education
Tlili, A., Shehata, B., Adarkwah, M. A., Bozkurt, A., Hickey, D. T., Huang, R., & Agyemang, B. (2023) · Smart Learning Environments · doi:10.1186/s40561-023-00237-x · Cited by 2,003
ChatGPT is fun, but not an author
Thorp, H. H. (2023) · Science · doi:10.1126/science.adg7879 · Cited by 1,437
Lim, W. M., Gunasekara, A., Pallant, J. L., Pallant, J. I., & Pechenkina, E. (2023) · The International Journal of Management Education · doi:10.1016/j.ijme.2023.100790 · Cited by 1,261
Grassini, S. (2023) · Education Sciences · doi:10.3390/educsci13070692 · Cited by 1,048
Strzelecki, A. (2024) · Interactive Learning Environments · doi:10.1080/10494820.2023.2209881 · Cited by 807
Using ChatGPT for second language writing: pitfalls and potentials
Barrot, J. S. (2023) · Assessing Writing · doi:10.1016/j.asw.2023.100745 · Cited by 770
Yilmaz, R., & Karaoglan Yilmaz, F. G. (2023) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2023.100147 · Cited by 745
Impact of ChatGPT on learners in a L2 writing practicum: an exploratory investigation
Yan, D. (2023) · Education and Information Technologies · doi:10.1007/s10639-023-11742-4 · Cited by 713
Engineering Education in the Era of ChatGPT: Promise and Pitfalls of Generative AI for Education
Qadir, J. (2023) · 2023 IEEE Global Engineering Education Conference (EDUCON) · doi:10.1109/educon54358.2023.10125121 · Cited by 707
A Conversation on Artificial Intelligence, Chatbots, and Plagiarism in Higher Education
King, M. R., & chatGPT (2023) · Cellular and Molecular Bioengineering · doi:10.1007/s12195-022-00754-8 · Cited by 602
Are We There Yet? - A Systematic Literature Review on Chatbots in Education
Wollny, S., Schneider, J., Di Mitri, D., Weidlich, J., Rittberger, M., & Drachsler, H. (2021) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2021.654924 · Cited by 557
Unlocking the opportunities through ChatGPT Tool towards ameliorating the education system
Javaid, M., Haleem, A., Singh, R. P., Khan, S., & Khan, I. H. (2023) · BenchCouncil Transactions on Benchmarks, Standards and Evaluations · doi:10.1016/j.tbench.2023.100115 · Cited by 535
Exploring Students’ Perceptions of ChatGPT: Thematic Analysis and Follow-Up Survey
Shoufan, A. (2023) · IEEE Access · doi:10.1109/access.2023.3268224 · Cited by 499
Yu, H. (2023) · Frontiers in Psychology · doi:10.3389/fpsyg.2023.1181712 · Cited by 409
Yu, H., & Guo, Y. (2023) · Frontiers in Education · doi:10.3389/feduc.2023.1183162 · Cited by 357
Nikolic, S., Daniel, S., Haque, R., Belkina, M., Hassan, G. M., Grundy, S., Lyden, S., Neal, P., & Sandison, C. (2023) · European Journal of Engineering Education · doi:10.1080/03043797.2023.2213169 · Cited by 336
Impact of ChatGPT on Learning Motivation: Teachers and Students' Voices
Ali, J. K. M., Shamsan, M. A. A., Hezam, T. A., & Mohammed, A. A. Q. (2023) · Journal of English Studies in Arabia Felix · doi:10.56540/jesaf.v2i1.51 · Cited by 335
Tiwari, C. K., Bhat, M. A., Khan, S. T., Subramaniam, R., & Khan, M. A. I. (2024) · Interactive Technology and Smart Education · doi:10.1108/itse-04-2023-0061 · Cited by 315
Kohnke, L., Moorhouse, B. L., & Zou, D. (2023) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2023.100156 · Cited by 309
Analysing the Role of ChatGPT in Improving Student Productivity in Higher Education
Fauzi, F., Tuhuteru, L., Sampe, F., Ausat, A. M. A., & Hatta, H. R. (2023) · Journal on Education · doi:10.31004/joe.v5i4.2563 · Cited by 283
Liu, M., Ren, Y., Nyagoga, L. M., Stonier, F., Wu, Z., & Yu, L. (2023) · Future in Educational Research · doi:10.1002/fer3.10 · Cited by 267
Using artificial intelligence in craft education: crafting with text-to-image generative models
Vartiainen, H., & Tedre, M. (2023) · Digital Creativity · doi:10.1080/14626268.2023.2174557 · Cited by 263
Exploring the integration of ChatGPT in education: adapting for the future
Elbanna, S., & Armstrong, L. (2024) · Management & Sustainability: An Arab Review · doi:10.1108/msar-03-2023-0016 · Cited by 238
Alyoussef, I. Y. (2023) · Heliyon · doi:10.1016/j.heliyon.2023.e13751 · Cited by 181
ChatGPT in physics education: A pilot study on easy-to-implement activities
Bitzenbauer, P. (2023) · Contemporary Educational Technology · doi:10.30935/cedtech/13176 · Cited by 181
The Role and Potential Contributions of the Artificial Intelligence Language Model ChatGPT
Berşe, S., Akça, K., Dirgar, E., & Kaplan Serin, E. (2024) · Annals of Biomedical Engineering · doi:10.1007/s10439-023-03296-w · Cited by 63
Seeing ChatGPT Through Students' Eyes: An Analysis of TikTok Data
Haensch, A.-C., Ball, S., Herklotz, M., & Kreuter, F. (2023) · arXiv · doi:10.48550/arxiv.2303.05349 · Cited by 62
MOOCs and upskilling in Australia: A qualitative literature study
Santandreu Calonge, D., Aman Shah, M., Riggs, K., Connor, M., & Wang, S. (2019) · Cogent Education · doi:10.1080/2331186x.2019.1687392 · Cited by 44
Cited by
Fan, Y., Tang, L., Le, H., Shen, K., Tan, S., Zhao, Y., Shen, Y., Li, X., & Gašević, D. (2025) · British Journal of Educational Technology · doi:10.1111/bjet.13544 · Cited by 805
Batista, J., Mesquita, A., & Carnaz, G. (2024) · Information · doi:10.3390/info15110676 · Cited by 256
ChatGPT in the higher education: A systematic literature review and research challenges
Baig, M. I., & Yadegaridehkordi, E. (2024) · International Journal of Educational Research · doi:10.1016/j.ijer.2024.102411 · Cited by 208
Does ChatGPT Help Novice Programmers Write Better Code? Results From Static Code Analysis
Haindl, P., & Weinberger, G. (2024) · IEEE Access · doi:10.1109/access.2024.3445432 · Cited by 45
“This aligns with findings from Chen et al. [28] and Rahman and Watanobe [30], which highlighted potential drawbacks of over-reliance on AI tools, such as reduced critical thinking and the risk of not developing independent problem-solving skills.”
MethodResult
Metadata
- Authors
- Chen, J., Zhuo, Z., & Lin, J.
- Venue
- Sustainability
- Publisher
- MDPI AG
- Published
- 2023-12-18
- Volume
- 15
- Issue
- 24
- Pages
- 16928
- ISSN
- 2071-1050 (ISSN-L)
- OpenAlex
- W4389894817
- Cited by
- 61
- References
- 37
- FWCI
- 14.33
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.