Introducing large language models in communication and public relations education: A mixed-methods pilot study
Open accessAbstract
This mixed-methods pilot study investigates the potential of large language models (LLMs), specifically ChatGPT 3.5, to assist communication and public relations students in improving academic writing, focusing on clarity, conciseness, and coherence—skills aligned with cognitive load theory. An analysis of 60 abstracts (30 student-drafted abstracts and 30 AI-augmented abstracts) through qualitative and quantitative methods revealed modest enhancements. Two blinded expert evaluators identified statistically significant improvements in clarity and conciseness (Evaluator 1: p = 0.0024; Evaluator 2: p = 0.0462), although evaluator variability highlighted assessment subjectivity. Without a control group, causal claims are limited, and training sessions may have enhanced students’ proficiency with ChatGPT, complicating attribution. The students perceived greater benefits (mean improvement = 1.2) than the evaluators confirmed (mean = 0.52), suggesting potential overconfidence. Risks, including overreliance on AI and threats to originality, underscore the need for ethical guidelines, such as critical evaluation training. Owing to its small, homogeneous sample size and short duration, this study lacks demographic diversity and longitudinal insight, thus necessitating broader, long-term research.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The development of critical thinking is gaining traction as a complementary skill to help students evaluate AI outputs and question their applicability.
“the development of critical thinking is gaining traction as a complementary skill, helping students to evaluate AI outputs and question their applicability (Cao et al., 2025; Cowling et al., 2023; Fitzek & B â rg ă oanu, 2025; Malik et al., 2023).”
From Six Institutional Intervention Areas to Support Ethical and Effective Student Use of Generative AI in Higher Education: A Narrative Review (2026) · cites this paper as Fitzek & Bârgăoanu, 2025
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Cognitive Load During Problem Solving: Effects on Learning
Sweller, J. (1988) · Cognitive Science · doi:10.1016/0364-0213(88)90023-7 · Cited by 11,048 · see citing works
Chatting and cheating: Ensuring academic integrity in the era of ChatGPT
Cotton, D. R. E., Cotton, P. A., & Shipway, J. R. (2024) · Innovations in Education and Teaching International · doi:10.1080/14703297.2023.2190148 · Cited by 3,087 · see citing works
Chatting and Cheating. Ensuring academic integrity in the era of ChatGPT
Cotton, D., Cotton, P., & Shipway, J. R. (2023) · doi:10.35542/osf.io/mrz8h · Cited by 3,079 · see citing works
The Cambridge Handbook of Multimedia Learning
(2005) · doi:10.1017/cbo9780511816819 · Cited by 1,595 · see citing works
Marzuki, Widiati, U., Rusdin, D., Darwin, & Indrawati, I. (2023) · Cogent Education · doi:10.1080/2331186x.2023.2236469 · Cited by 524 · cited 5× · see citing works
“Although recent research suggests that LLMs can improve such features (Imran & Almusharraf, 2023; Marzuki et al., 2023; Montenegro-Rueda et al., 2023), questions remain about authenticity, intellectual ownership, and the balance between AI-generated enhancements and a student’s own voice (Castro,…”
InfluentialImpact of the Implementation of ChatGPT in Education: A Systematic Review
Montenegro-Rueda, M., Fernández-Cerero, J., Fernández-Batanero, J. M., & López-Meneses, E. (2023) · Computers · doi:10.3390/computers12080153 · Cited by 509 · cited 4× · see citing works
“Unlike intelligent tutoring systems or adaptive learning platforms, ChatGPT is a pattern-based genera - tive model that relies on extensive training data rather than learner-specific instructions (Introducing ChatGPT, 2024; Montenegro-Rueda et al., 2023; Roll & Wylie, 2016).”
InfluentialImran, M., & Almusharraf, N. (2023) · Contemporary Educational Technology · doi:10.30935/cedtech/13605 · Cited by 421 · see citing works
Creative Writing with an AI-Powered Writing Assistant: Perspectives from Professional Writers
Ippolito, D., Yuan, A., Coenen, A., & Burnam, S. (2022) · arXiv · doi:10.48550/arxiv.2211.05030 · Cited by 52 · see citing works
Flagged first, then by citations
Research works printed but not linked: 21
- Alawaad, H. (2021). The role of artificial intelligence (AI) in public relations and product marketing in modern organizations. Turkish Journal of Computer and Mathematics Education (TURCOMAT), 12, 3180-3187.
- Azer, S. A. (2017). Social media channels in health care research and rising ethical issues. AMA Journal of Ethics, 19(11), 1061-1069. h t t p s : / / j o
- Bayat, N. (2014). The effect of the process writing approach on writing success and anxiety. Educational Sciences: Theory and Practice, 14(3), 1133-1141. https://eric.ed.gov/?id=EJ1034097
- Charney, D. (1984). The validity of using holistic scoring to evaluate writing: A critical overview. Research in the Teaching of English, 18(1), 65-81. https://doi.org/10.58680/rte198415687 · DOI
- de Castro, C. A. (2023). A discussion about the impact of ChatGPT in education: Benefits and concerns. Journal of Business Theory and Practice, 11(2). https://doi.org/10.22158/jbtp.v11n2p28. Article 2. · DOI · Paper page
- Dergaa, I., Chamari, K., Zmijewski, P., & Saad, H. B. (2023). From human writing to artificial intelligence generated text: Examining the prospects and potential threats of ChatGPT in academic writing. Biol- ogy of Sport, 40(2), 615-622. h t t p s : / / d o i . o r g / 1 0 . 5 1 1 4 / b i o l s p o r t . 2 3 . 1 2 5 6 · DOI · Paper page
- Esman, E. N., Madrigal, D. V., & Mascuňana, C. G. (2021). Social media exposure and english writing proficiency of grade 11 students in a Philippine public high school. Technium Social Sciences Jour- nal, 20, 212-232. https://doi.org/10.47577/tssj.v20i1.3001 · DOI
- Eysenbach, G. (2004). Improving the quality of web surveys: The checklist for reporting results of internet E-Surveys (CHERRIES). Journal of Medical Internet Research, 6(3), e132. h t t p s : / / d o i . o r g / 1 0 . 2 1 9 6 / j m i r . 6 . 3 . e 3 4 · DOI · Paper page
- Hashemi, M. R., & Gohari Moghaddam, I. (2019). A mixed methods genre analysis of the discussion sec- tion of MMR articles in applied linguistics. Journal of Mixed Methods Research, 13(2), 242-260. https://doi.org/10.1177/1558689816674626 · DOI
- Jeon, J., & Lee, S. (2023). Large Language models in education: A focus on the complementary rela- tionship between human teachers and ChatGPT. Education and Information Technologies, 28(12), 15873-15892. https://doi.org/10.1007/s10639-023-11834-1 · DOI · Paper page
- Kellogg, R. T., & Whiteford, A. P. (2009). Training advanced writing skills: The case for deliberate prac- tice. Educational Psychologist, 44(4), 250-266. https://doi.org/10.1080/00461520903213600 · DOI
- Kouam, A. W. F., & Muchowe, R. M. (2024). Exploring graduate students' perception and adoption of AI chatbots in Zimbabwe: Balancing pedagogical innovation and development of higher-order cogni- tive skills. Journal of Applied Learning and Teaching, 7(1). https://doi.org/10.37074/jalt.2024.7.1.12 · DOI
- Li, Q., Fu, L., Zhang, W., Chen, X., Yu, J., Xia, W., Zhang, W., Tang, R., & Yu, Y. (2024). Adapting large Language models for education: Foundational capabilities, potentials, and challenges. ArXiv. h t t p s : / / d o i . o r g / 1 0 . 4 8 5 5 0 / A r X i v . 2 4 0 1 . 0 8 6 6 4 . No. ArXiv:2401.08664. · DOI
- Lydia, E. G., Vidhyavathi, P., Malathi, P., & AI IN EDUCATION: OPPORTUNITIES AND CHAL- LENGES FOR PERSONALIZED LEARNING. (2023). A STUDY ON. Industrial Engineering Journal, 52(05), 750-759. h t t p s : / / d o i . o r g / 1 0 . 3 6 8 9 3 / I E J . 2 0 2 3 . V 5 2 I 0 5 . 7 5 0 -7 5 9 · DOI
- Mayer, C. W. F., Ludwig, S., & Brandt, S. (2023). Prompt text classifications with transformer models! An exemplary introduction to prompt-based learning with large Language models. Journal of Research on Technology in Education, 55(1), 125-141. h t t p s : / / d o i . o r g / 1 0 . 1 0 8 0 / 1 5 3 9 1 5 2 3 . 2 0 2 2 . 2 1 4 2 8 · DOI
- Roll, I., & Wylie, R. (2016). Evolution and revolution in artificial intelligence in education. International Journal of Artificial Intelligence in Education, 26(2), 582-599. h t t p s : / / d o i . o r g / 1 0 . 1 0 0 7 / s 4 0 5 9 3 -0 1 6 -0 1 1 0 -3 · DOI · Paper page
- Schaffler, Y., Probst, T., Jesser, A., Humer, E., Pieh, C., Stippl, P., Haid, B., & Schigl, B. (2022). Perceived barriers and facilitators to psychotherapy utilization and how they relate to patient's psychotherapeu- tic goals. Healthcare, 10(11), 2228. https://doi.org/10.3390/healthcare10112228 · DOI
- Shibani, A., Knight, S., Kitto, K., Karunanayake, A., & Buckingham Shum, S. (2024). Untangling Critical Interaction with AI in Students' Written Assessment. Extended Abstracts of the CHI Conference on Human Factors in Computing Systems, 1-6. https://doi.org/10.1145/3613905.3651083 · DOI · Paper page
- Tan, X. (2023). The impact of ChatGPT on education and future prospects. Highlights in science. Engi- neering and Technology, 61, 138-143. https://doi.org/10.54097/hset.v61i.10285 · DOI
- Tashakkori, A., & Teddlie, C. (2010). SAGE handbook of mixed methods in social & behavioral research. SAGE.
- Workspace, G. (n.d.) (Ed.). (2025). Google Forms: Online Form Builder. Google Workspace. Retrieved March 15, from https://workspace.google.com/products/forms/ Кочубей, В. (2021). Труднощі Написання академічного Есе Та Шляхи Їх Подолання. Науковий Вісник Чернівецького Національного Університету Імені Юрія Федьковича. Серія: Германська Філологія, 831-832, Article 831-832. h t t p s : / / d o i . o r g / 1 0 . 3 1 8 6 1 / g p h 2 0 2 1 . 8 3 1 -8 3 2 . 1 4 7 -1 5 6 · DOI
Cited by
Sorted by citations
Jayasinghe, S., Gamage, K. A. A., Yang, D., Cheng, C., Disanayake, C., & Apeji, U. D. (2026) · Education Sciences · doi:10.3390/educsci16010137 · Cited by 23 · see citing works
- Authors
- Fitzek, S., & Bârgăoanu, A.
- Affiliations
- National School of Political Science and Public Administration (RO)
- Corresponding author
- Fitzek, Sebastian
- Venue
- International Journal of Artificial Intelligence in Education
- Publisher
- Elsevier BV
- Published
- 2025-12
- Volume
- 35
- Issue
- 4
- Pages
- 2478-2494
- ISSN
- 1560-4292 (ISSN-L)
- OpenAlex
- W4409873783
- Cited by
- 7
- References
- 8
- Citations by year
- 2026: 6 · 2025: 1
- FWCI
- 7.34
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- International Journal of Artificial Intelligence in Education (publishedVersion, cc-by)
- Topics
- Physical Sciences › Computer Science › Information Systems › Artificial Intelligence in Education
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Social Sciences › Arts and Humanities › History and Philosophy of Science › Academic Writing and Publishing
- Keywords
- large language models, ChatGPT, academic writing, communication education, public relations education, cognitive load theory, clarity, brevity, coherence, AI-assisted writing, overconfidence, overreliance on AI, originality, AI ethics in education
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
