Unleashing ChatGPT's Power: A Case Study on Optimizing Information Retrieval in Flipped Classrooms via Prompt Engineering
The DOI matches its Crossref record; 62 of the 62 references it cites were found in a registry or an index.
Abstract
This research project investigates the impact of prompt engineering, a key aspect of chat generative pretrained transformer (ChatGPT), on college students' information retrieval in flipped classrooms. In recent years, an increasing number of students have been using AI-based tools, such as ChatGPT rather than traditional research engines to learn and to complete course assignments. Despite this growing trend, previous research has largely overlooked the influence of prompt engineering on students' use of ChatGPT and effective strategies for improving the quality of information retrieval in learning settings. To address this research gap, this study examines the information quality obtained from ChatGPT in a flipped classroom by evaluating its effectiveness in task completion among 26 novice undergraduates from the same major and cohort. The experimental results provide evidence that proficient mastery of prompt engineering improves the quality of information obtained by students using ChatGPT. Consequently, by acquiring proficiency in prompt engineering, students can maximize the positive impact of ChatGPT, obtain high-quality information, and enhance their learning efficiency in flipped classrooms.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929
ChatGPT for Good? On Opportunities and Challenges of Large Language Models for Education
Kasneci, E., Seßler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., et al. (2023) · doi:10.35542/osf.io/5er8f · Cited by 7,552
ChatGPT for good? On opportunities and challenges of large language models for education
Kasneci, E., Sessler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., et al. (2023) · Learning and Individual Differences · doi:10.1016/j.lindif.2023.102274 · Cited by 7,326 · cited 1×
“Assuming the role of a virtual tutor, ChatGPT assists students in online independent learning by addressing their queries [68], while also bolstering group dynamics through the provision of discussion structures and real-time feedback [69].”
BackgroundDwivedi, Y. K., Kshetri, N., Hughes, L., Slade, E. L., Jeyaraj, A., Kar, A. K., Baabdullah, A. M., Koohang, A., Raghavan, V., Ahuja, M., et al. (2023) · International Journal of Information Management · doi:10.1016/j.ijinfomgt.2023.102642 · Cited by 4,584
Chain-Of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q. V., & Zhou, D. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1800 · Cited by 4,505
Liu, P., Yuan, W., Fu, J., Jiang, Z., Hayashi, H., & Neubig, G. (2023) · ACM Computing Surveys · doi:10.1145/3560815 · Cited by 4,361
Liu, P., Yuan, W., Fu, J., Jiang, Z., Hayashi, H., & Neubig, G. (2021) · arXiv · doi:10.48550/arxiv.2107.13586 · Cited by 4,352 · cited 2×
“In line with other research findings [83], [84], [87], [88], which prompt engineering plays a pivotal role in efficiently conveying users’ intentions to ChatGPT, our study presents an exploration of this assertion within the realm of education, serving to enlighten educators on the use of prompt…”
BackgroundResultKung, T. H., Cheatham, M., Medenilla, A., Sillos, C., De Leon, L., Elepaño, C., Madriaga, M., Aggabao, R., Diaz-Candido, G., Maningo, J., et al. (2023) · PLOS Digital Health · doi:10.1371/journal.pdig.0000198 · Cited by 4,115 · cited 1×
“However, when applied in fields, such as law [33], [48] and medical education [49], [50], its performance is about average.”
BackgroundKung, T. H., Cheatham, M., ChatGPT, Medenilla, A., Sillos, C., De Leon, L., Elepaño, C., Madriaga, M., Aggabao, R., Diaz-Candido, G., et al. (2022) · doi:10.1101/2022.12.19.22283643 · Cited by 4,115
Sallam, M. (2023) · Healthcare · doi:10.3390/healthcare11060887 · Cited by 3,126
Sallam, M. (2023) · doi:10.1101/2023.02.19.23286155 · Cited by 3,123 · cited 1×
“Furthermore, in a comprehensive review of 60 articles encompassing various academic disciplines, Sallam [74] identified the challenges linked to the use of ChatGPT in education, specifically in relation to accuracy and reliability.”
BackgroundTraining Language Models to Follow Instructions with Human Feedback
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-2011 · Cited by 3,117
Chatting and cheating: Ensuring academic integrity in the era of ChatGPT
Cotton, D. R. E., Cotton, P. A., & Shipway, J. R. (2024) · Innovations in Education and Teaching International · doi:10.1080/14703297.2023.2190148 · Cited by 3,087
Chatting and Cheating. Ensuring academic integrity in the era of ChatGPT
Cotton, D., Cotton, P., & Shipway, J. R. (2023) · doi:10.35542/osf.io/mrz8h · Cited by 3,079
Baidoo-Anu, D., & Owusu Ansah, L. (2023) · doi:10.2139/ssrn.4337484 · Cited by 3,004 · cited 2×
“This limitation becomes particularly worrisome when considering ChatGPT’s potential to generate incorrect or false information [47], [76], [77], posing a significant risk for students who heavily depend on ChatGPT as a source of information.”
BackgroundPaLM: Scaling Language Modeling with Pathways
Chowdhery, A., Narang, S., Devlin, J., Bosma, M., Mishra, G., Roberts, A., Barham, P., Chung, H. W., Sutton, C., Gehrmann, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.02311 · Cited by 2,590
Hierarchical Text-Conditional Image Generation with CLIP Latents
Ramesh, A., Dhariwal, P., Nichol, A., Chu, C., & Chen, M. (2022) · arXiv · doi:10.48550/arxiv.2204.06125 · Cited by 2,511 · cited 1×
“1) Transformer Architecture: ChatGPT is built upon the powerful foundation of the Transformer architecture, serving as the backbone for numerous state-of-the-art models [1], such as GPT-3 [25], DALL-E-2 [35], Codex [36], and Gopher [37].”
MethodGilson, A., Safranek, C. W., Huang, T., Socrates, V., Chi, L., Taylor, R. A., & Chartash, D. (2023) · JMIR Medical Education · doi:10.2196/45312 · Cited by 2,078
What Is the Impact of ChatGPT on Education? A Rapid Review of the Literature
Lo, C. K. (2023) · Education Sciences · doi:10.3390/educsci13040410 · Cited by 2,024
Large language models are zero-shot reasoners
Kojima, T., Gu, S. (., Reid, M., Matsuo, Y., & Iwasawa, Y. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1613 · Cited by 2,013
Large Language Models are Zero-Shot Reasoners
Kojima, T., Gu, S. S., Reid, M., Matsuo, Y., & Iwasawa, Y. (2022) · arXiv · doi:10.48550/arxiv.2205.11916 · Cited by 2,013
What if the devil is my guardian angel: ChatGPT as a case study of using chatbots in education
Tlili, A., Shehata, B., Adarkwah, M. A., Bozkurt, A., Hickey, D. T., Huang, R., & Agyemang, B. (2023) · Smart Learning Environments · doi:10.1186/s40561-023-00237-x · Cited by 2,003 · cited 1×
“A study has shown that ChatGPT is a valuable tool for educators, helping them identify essential curriculum content, and providing an outline [58].”
BackgroundChatGPT: five priorities for research
van Dis, E. A. M., Bollen, J., Zuidema, W., van Rooij, R., & Bockting, C. L. (2023) · Nature · doi:10.1038/d41586-023-00288-7 · Cited by 1,842
ChatGPT: Bullshit spewer or the end of traditional assessments in higher education?
(2023) · Journal of Applied Learning & Teaching · doi:10.37074/jalt.2023.6.1.9 · Cited by 1,806 · cited 2×
“For instance, Rudolph et al. [64] proposed the use of a flipped classroom approach, wherein students are expected to study preclass materials in preparation for the class.”
BackgroundChatGPT is fun, but not an author
Thorp, H. H. (2023) · Science · doi:10.1126/science.adg7879 · Cited by 1,437
Evaluating Large Language Models Trained on Code
Chen, M., Tworek, J., Jun, H., Yuan, Q., Pinto, H. P. D. O., Kaplan, J., Edwards, H., Burda, Y., Joseph, N., Brockman, G., et al. (2021) · arXiv · doi:10.48550/arxiv.2107.03374 · Cited by 1,436
A SWOT analysis of ChatGPT: Implications for educational practice and research
Farrokhnia, M., Banihashem, S. K., Noroozi, O., & Wals, A. (2024) · Innovations in Education and Teaching International · doi:10.1080/14703297.2023.2195846 · Cited by 1,384
Dave, T., Athaluri, S. A., & Singh, S. (2023) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2023.1169595 · Cited by 1,204
Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigm
Reynolds, L., & McDonell, K. (2021) · Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411763.3451760 · Cited by 952 · cited 1×
“To guide models in generating effective natural language prompts, Reynolds et al. [89] introduced the concept of meta-prompts.”
ChatGPT User Experience: Implications for Education
Zhai, X. (2023) · doi:10.2139/ssrn.4312418 · Cited by 935 · cited 1×
“Zhai[79]proposedtheexplorationofinnovative formatstofosterstudents’creativeandcriticalthinking.”
Perkins, M. (2023) · Journal of University Teaching and Learning Practice · doi:10.53761/1.20.02.07 · Cited by 839
A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT
White, J., Fu, Q., Hays, S., Sandborn, M., Olea, C., Gilbert, H., Elnashar, A., Spencer-Smith, J., & Schmidt, D. C. (2023) · arXiv · doi:10.48550/arxiv.2302.11382 · Cited by 838
Prompt Engineering with ChatGPT: A Guide for Academic Writers
Giray, L. (2023) · Annals of Biomedical Engineering · doi:10.1007/s10439-023-03272-4 · Cited by 776 · cited 1×
“[92], academic writers can adapt to the ever-changing landscape by acquiring expertise in prompt engineering and harnessing the power of large-scale language models.”
BackgroundOpen AI in Education, the Responsible and Ethical Use of ChatGPT Towards Lifelong Learning
Mhlanga, D. (2023) · Sustainable Development Goals Series · doi:10.1007/978-3-031-37776-1_17 · Cited by 718
Engineering Education in the Era of ChatGPT: Promise and Pitfalls of Generative AI for Education
Qadir, J. (2023) · 2023 IEEE Global Engineering Education Conference (EDUCON) · doi:10.1109/educon54358.2023.10125121 · Cited by 707 · cited 2×
“This limitation becomes particularly worrisome when considering ChatGPT’s potential to generate incorrect or false information [47], [76], [77], posing a significant risk for students who heavily depend on ChatGPT as a source of information.”
BackgroundDesign Guidelines for Prompt Engineering Text-to-Image Generative Models
Liu, V., & Chilton, L. B. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3501825 · Cited by 639 · cited 1×
“Numerous studies have explored the impact of prompt engineering on AI generative models across domains, including image generation tasks [85] and classification tasks [86], highlighting the critical role of excellent prompt design for LLMs like ChatGPT [87], [88].”
BackgroundDesign Guidelines for Prompt Engineering Text-to-Image Generative Models
Liu, V., & Chilton, L. B. (2021) · arXiv · doi:10.48550/arxiv.2109.06977 · Cited by 635
ChatGPT - Reshaping medical education and clinical management
Khan, R. A., Jawaid, M., Khan, A. R., & Sajjad, M. (2023) · Pakistan Journal of Medical Sciences · doi:10.12669/pjms.39.2.7653 · Cited by 628
Xu, L., Sanders, L., Li, K., & Chow, J. C. L. (2021) · JMIR Cancer · doi:10.2196/27850 · Cited by 571 · cited 1×
“These AI-based chatbots have proven to be invaluable assets in various sectors, such as customer service [5] and healthcare [6], as they can optimize workflow efficiencies, minimize expenses, and elevate the overall user experience.”
BackgroundAre We There Yet? - A Systematic Literature Review on Chatbots in Education
Wollny, S., Schneider, J., Di Mitri, D., Weidlich, J., Rittberger, M., & Drachsler, H. (2021) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2021.654924 · Cited by 557 · cited 1×
“Although there are many studies on flipped classrooms and AI-based chatbots [14], [22], [23], few have focused on how to improve students’ skills in using chatbots to obtain high-quality responses, thus highlighting the need for further research in this area.”
BackgroundCheng, L., Ritzhaupt, A. D., & Antonenko, P. (2019) · Educational Technology Research and Development · doi:10.1007/s11423-018-9633-7 · Cited by 494
PTR: Prompt Tuning with Rules for Text Classification
Han, X., Zhao, W., Ding, N., Liu, Z., & Sun, M. (2021) · arXiv · doi:10.48550/arxiv.2105.11259 · Cited by 478 · cited 1×
“Numerous studies have explored the impact of prompt engineering on AI generative models across domains, including image generation tasks [85] and classification tasks [86], highlighting the critical role of excellent prompt design for LLMs like ChatGPT [87], [88].”
BackgroundCao, Y., Li, S., Liu, Y., Yan, Z., Dai, Y., Yu, P. S., & Sun, L. (2023) · arXiv · doi:10.48550/arxiv.2303.04226 · Cited by 463
Mathematical Capabilities of ChatGPT
Frieder, S., Pinchetti, L., Chevalier, A., Griffiths, R.-R., Salvatori, T., Lukasiewicz, T., Petersen, P. C., & Berner, J. (2023) · arXiv · doi:10.48550/arxiv.2301.13867 · Cited by 385
Welcome to the Era of ChatGPT et al.: The Prospects of Large Language Models
Teubner, T., Flath, C. M., Weinhardt, C., van der Aalst, W., & Hinz, O. (2023) · Business & Information Systems Engineering · doi:10.1007/s12599-023-00795-x · Cited by 384 · cited 1×
“ChatGPT has attracted an impressive 100 million active users in a mere two months since its inception [34].”
BackgroundSolving Quantitative Reasoning Problems with Language Models
Lewkowycz, A., Andreassen, A., Dohan, D., Dyer, E., Michalewski, H., Ramasesh, V., Slone, A., Anil, C., Schlag, I., Gutman-Solo, T., et al. (2022) · arXiv · doi:10.48550/arxiv.2206.14858 · Cited by 358 · cited 1×
“Google Research introduced Minerva [30], an advanced model that builds upon the PaLM [31] while incorporating a science-and-math-focused dataset.”
Method(2022) · doi:10.52202/068431-0278 · Cited by 358
Large Language Models Are Human-Level Prompt Engineers
Zhou, Y., Muresanu, A. I., Han, Z., Paster, K., Pitis, S., Chan, H., & Ba, J. (2022) · arXiv · doi:10.48550/arxiv.2211.01910 · Cited by 330
Nicolescu, L., & Tudorache, M. T. (2022) · Electronics · doi:10.3390/electronics11101579 · Cited by 280 · cited 1×
“These AI-based chatbots have proven to be invaluable assets in various sectors, such as customer service [5] and healthcare [6], as they can optimize workflow efficiencies, minimize expenses, and elevate the overall user experience.”
BackgroundChatGPT passing USMLE shines a spotlight on the flaws of medical education
Mbakwe, A. B., Lourentzou, I., Celi, L. A., Mechanic, O. J., & Dagan, A. (2023) · PLOS Digital Health · doi:10.1371/journal.pdig.0000205 · Cited by 273
ChatGPT and Software Testing Education: Promises & Perils
Jalil, S., Rafi, S., LaToza, T. D., Moran, K., & Lam, W. (2023) · 2023 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW) · doi:10.1109/icstw58534.2023.00078 · Cited by 270 · cited 1×
“In contrast, ChatGPT’s performance in mathematics [51], sports science [52], and software testing [53] is notably poor.”
BackgroundHew, K. F., Huang, W., Du, J., & Jia, C. (2023) · Journal of Computing in Higher Education · doi:10.1007/s12528-022-09338-x · Cited by 245 · cited 1×
“Many studies have shown that AI-based chatbots can play a variety of roles, such as learning chatbots [15], [16], [17], assistant chatbots [18], [19], and mentor chatbots [20], [21].”
BackgroundChatgpt for next generation science learning
Zhai, X. (2023) · XRDS Crossroads The ACM Magazine for Students · doi:10.1145/3589649 · Cited by 178 · cited 1×
“According to Zhai [59], ChatGPT provides valuable suggestions in the context of special education, which he described as being particularly advantageous for students with unique learning requirements.”
BackgroundPAL: Program-aided Language Models
Gao, L., Madaan, A., Zhou, S., Alon, U., Liu, P., Yang, Y., Callan, J., & Neubig, G. (2022) · arXiv · doi:10.48550/arxiv.2211.10435 · Cited by 155 · cited 1×
“…of prompt engineering highlighted in numerous studies and the exploration of methods to elicit high-quality responses from LLMs [25], [26], [27], [28], [29], there remains a research gap regarding the impact of prompt engineering on students’ participation in ChatGPT-enabled flipped…”
BackgroundWill ChatGPT Get You Caught? Rethinking of Plagiarism Detection
Khalil, M., & Er, E. (2023) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-34411-4_32 · Cited by 146
Designing and evaluating three chatbot-enhanced activities for a flipped graduate course
Huang, W. (2019) · International Journal of Mechanical Engineering and Robotics Research · doi:10.18178/ijmerr.8.5.813-818 · Cited by 62
Is ChatGPT a Good Tool for T&CM Students in Studying Pharmacology?
Nisar, S., & Aslam, M. S. (2023) · doi:10.2139/ssrn.4324310 · Cited by 60 · cited 1×
“Assuming the role of a virtual tutor, ChatGPT assists students in online independent learning by addressing their queries [68], while also bolstering group dynamics through the provision of discussion structures and real-time feedback [69].”
BackgroundChatGPT for Next Generation Science Learning
Zhai, X. (2023) · doi:10.2139/ssrn.4331313 · Cited by 58
An AI-based Approach for Grading Students’ Collaboration
Tomić, B. B., Kijevčanin, A. D., Ševarac, Z. V., & Jovanović, J. M. (2023) · IEEE Transactions on Learning Technologies · doi:10.1109/tlt.2022.3225432 · Cited by 37
‘Words are Flowing Out Like Endless Rain Into a Paper Cup’: ChatGPT & Law School Assessments
Hargreaves, S. (2023) · Legal Education Review · doi:10.53300/001c.83297 · Cited by 31
New Challenges in Scientific Publications: Referencing, Artificial Intelligence and ChatGPT
Švab, I., Klemenc-Ketiš, Z., & Zupanič, S. (2023) · Slovenian Journal of Public Health · doi:10.2478/sjph-2023-0015 · Cited by 29
Cited by
The Influence of ChatGPT on Student Engagement: A Systematic Review and Future Research Agenda
Lo, C. K., Hew, K. F., & Jong, M. S.-Y. (2024) · Computers & Education · doi:10.1016/j.compedu.2024.105100 · Cited by 392
Kong, S.-C., & Yang, Y. (2024) · IEEE Transactions on Learning Technologies · doi:10.1109/tlt.2024.3392830 · Cited by 298
“In K–12 education, immediate feedback can help students to quickly identify areas for improvement and understand complex concepts [21].”
BackgroundImplementing generative AI (GenAI) in higher education: A systematic review of case studies
Belkina, M., Daniel, S., Nikolic, S., Haque, R., Lyden, S., Neal, P., Grundy, S., & Hassan, G. M. (2025) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2025.100407 · Cited by 183
Liu, X., & Zhong, B. (2025) · Educational Research Review · doi:10.1016/j.edurev.2025.100741 · Cited by 78
Transforming education with generative AI (GAI): key insights and future prospects
Lang, Q., Wang, M., Yin, M., Liang, S., & Song, W. (2025) · IEEE Transactions on Learning Technologies · doi:10.1109/tlt.2025.3537618 · Cited by 76
Self-citation
Metadata
- Authors
- Wang, M., Wang, M., Xu, X., Yang, L., Cai, D., & Yin, M.
- Venue
- IEEE Transactions on Learning Technologies
- Published
- 2024
- Volume
- 17
- Pages
- 629-641
- ISSN
- 1939-1382 (ISSN-L)
- 2372-0050
- OpenAlex
- W4387665600
- Cited by
- 156
- References
- 62
- FWCI
- 36.66
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.