Evaluating the Longitudinal Effects of AI-Enhanced Collaborative Dialogue Modes on Computational Thinking and Language Proficiency in EFL Learners: A Mixed-Methods Approach
Abstract
This mixed-methods study compared the effectiveness of three AI-powered collaborative dialogue approaches—guided feedback (GF), interactive Q&A (IQ), and inquiry-driven scaffolding (IS)—on the computational thinking (CT) and language proficiency of 456 EFL learners. The triple-blind, randomized controlled trial was based on sociocultural theory and modern language learning models. We used advanced statistical growth modeling, analyzed conversation networks, and reviewed AI interaction logs. Participants (CEFR B1–B2; M age = 25.25) were divided into four groups (GF, IQ, IS, and control), with initial similarities confirmed through careful statistical checks. Results showed that the guided feedback group made the largest and most lasting gains in CT and language, significantly outperforming the control group and retaining skills over time. The IQ and IS groups improved moderately but had more difficulty retaining skills and applying them creatively. Qualitative analysis highlighted that GF helped learners monitor their thinking and correct mistakes, while the other AI methods led to higher mental effort and over-reliance on technology. The findings support AI designs that balance structured feedback with learner control; however, as this study was conducted exclusively with Chinese EFL learners, we strongly recommend cross-cultural replications to validate these findings and ensure fair, effective implementation beyond this specific cultural context.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Cites
Using thematic analysis in psychology
Braun, V., & Clarke, V. (2006) · Qualitative Research in Psychology · doi:10.1191/1478088706qp063oa · Cited by 194,414 · see citing works
Faul, F., Erdfelder, E., Lang, A.-G., & Buchner, A. (2007) · Behavior Research Methods · doi:10.3758/bf03193146 · Cited by 65,971 · see citing works
Lakens, D. (2013) · Frontiers in Psychology · doi:10.3389/fpsyg.2013.00863 · Cited by 10,303 · see citing works
Wing, J. M. (2006) · Communications of the ACM · doi:10.1145/1118178.1118215 · Cited by 8,102 · see citing works
Human Memory: A Proposed System and its Control Processes
Atkinson, R. C., & Shiffrin, R. M. (1968) · Psychology of Learning and Motivation · doi:10.1016/s0079-7421(08)60422-3 · Cited by 7,216 · see citing works
Toward an Emotion Nuanced Sensemaking Perspective of Organizational Consultation
Lundberg, C. C. (2000) · Emerald Group Publishing Limited eBooks · doi:10.1108/978-1-60752-837-120251015 · Cited by 3,340 · see citing works
A validity and reliability study of the computational thinking scales (CTS)
Korkmaz, Ö., Çakır, R., & Özden, M. Y. (2017) · Computers in Human Behavior · doi:10.1016/j.chb.2017.01.005 · Cited by 719 · see citing works
Zhai, C., & Wibowo, S. (2023) · Computers and Education Artificial Intelligence · doi:10.1016/j.caeai.2023.100134 · Cited by 363 · see citing works
Ebadi, S., & Amini, A. (2022) · Interactive Learning Environments · doi:10.1080/10494820.2022.2096638 · Cited by 235 · see citing works
The Computational Thinking Scale for Computer Literacy Education
Tsai, M., Liang, J., & Hsu, C.-Y. (2020) · Journal of Educational Computing Research · doi:10.1177/0735633120972356 · Cited by 229 · see citing works
Xia, Y., Shin, S.-Y., & Kim, J.-C. (2024) · Applied Sciences · doi:10.3390/app14135651 · Cited by 200 · see citing works
Hao, X., Demir, E., & Eyers, D. R. (2024) · Technology in Society · doi:10.1016/j.techsoc.2024.102662 · Cited by 183 · see citing works
Fathi, J., & Rahimi, M. (2024) · Computer Assisted Language Learning · doi:10.1080/09588221.2024.2374772 · Cited by 161 · see citing works
Huang, X., & Qiao, C. (2024) · Science & Education · doi:10.1007/s11191-022-00392-6 · Cited by 116 · see citing works
Scaffolding Computational Thinking With ChatGPT
Liao, J., Zhong, L., Zhe, L., Xu, H., Liu, M., & Xie, T. (2024) · IEEE Transactions on Learning Technologies · doi:10.1109/tlt.2024.3392896 · Cited by 113 · see citing works
AI-powered learning application use and gratification: an integrative model
Chang, Y., Lee, S., Wong, S. F., & Jeong, S.-P. (2021) · Information Technology and People · doi:10.1108/itp-09-2020-0632 · Cited by 102 · see citing works
Yuan, Y. (2024) · Interactive Learning Environments · doi:10.1080/10494820.2023.2282112 · Cited by 101 · see citing works
Weng, X., Ye, H., Dai, Y., & Ng, O. (2024) · Journal of Educational Computing Research · doi:10.1177/07356331241248686 · Cited by 100 · see citing works
Yan, W., Nakajima, T., & Sawada, R. (2024) · Education Sciences · doi:10.3390/educsci14040433 · Cited by 78 · see citing works
Bahari, A. (2018) · Computers & Education · doi:10.1016/j.compedu.2018.11.009 · Cited by 78 · see citing works
Enhancing English writing and higher-order thinking skills through computational thinking
Wu, T., Silitonga, L. M., & Murti, A. T. (2024) · Computers & Education · doi:10.1016/j.compedu.2024.105012 · Cited by 60 · see citing works
Tan, S. C., Chen, W., & Chua, B. L. (2023) · Learning Research and Practice · doi:10.1080/23735082.2023.2258895 · Cited by 55 · see citing works
What does AI bring to second language writing? A systematic review (2014-2024)
Feng, H., Li, K., & Zhang, L. J. (2025) · Language Learning & Technology · doi:10.64152/10125/73629 · Cited by 51 · see citing works
Rao, T. S. S., & Bhagat, K. K. (2024) · Educational Technology Research and Development · doi:10.1007/s11423-024-10364-y · Cited by 36 · see citing works
Wu, T., Asmara, A., Huang, Y., & Hapsari, I. P. (2024) · SAGE Open · doi:10.1177/21582440241249897 · Cited by 33 · see citing works
Xue, Y. (2024) · Education and Information Technologies · doi:10.1007/s10639-024-12596-0 · Cited by 33 · see citing works
Fostering L2 Learners’ Pronunciation and Motivation via Affordances of Artificial Intelligence
Rad, H. S., & Roohani, A. (2024) · Computers in the Schools · doi:10.1080/07380569.2024.2330427 · Cited by 32 · see citing works
Shin, Y., Jung, J., Choi, S., & Jung, B. (2024) · Education and Information Technologies · doi:10.1007/s10639-024-13104-0 · Cited by 31 · see citing works
Shang, S., & Geng, S. (2024) · Journal of Computer Assisted Learning · doi:10.1111/jcal.12996 · Cited by 29 · see citing works
Advancements in English Language Teaching: Harnessing the Power of Artificial Intelligence
Umar, U. (2024) · FLIP Foreign Language Instruction Probe · doi:10.54213/flip.v3i1.402 · Cited by 28 · see citing works
Developing and assessing second language listening and speaking: Does AI make it better?
Goh, C. C. M., & Aryadoust, V. (2025) · Annual Review of Applied Linguistics · doi:10.1017/s0267190525100111 · Cited by 27 · see citing works
Hu, X., He, W., Chiu, T. K. F., & Zhao, L. (2022) · Education and Information Technologies · doi:10.1007/s10639-022-11556-w · Cited by 24 · see citing works
Qin, Z., & Chuaychoowong, M. (2025) · Language Teaching Research Quarterly · doi:10.32038/ltrq.2025.49.04 · Cited by 15 · see citing works
Integration of AI-Technologies into ELT: A Brief Study
Tulasi, L., & Rao, C. S. (2023) · Journal for Research Scholars and Professionals of English Language Teaching · doi:10.54850/jrspelt.7.38.003 · Cited by 13 · see citing works
Crum, S., Li, B., & Kou, X. (2024) · Communications in computer and information science · doi:10.1007/978-3-031-61953-3_6 · Cited by 12 · see citing works
Li, D., & Zhao, Y. (2025) · The Asia-Pacific Education Researcher · doi:10.1007/s40299-025-01023-8 · Cited by 11 · see citing works
Enriching the Teaching-Learning Experience by Using AI Tools in the L2 Classroom
Barrios-Beltran, D. (2024) · Advances in educational technologies and instructional design book series · doi:10.4018/979-8-3693-0872-1.ch004 · Cited by 8 · see citing works
Kim, E., & Dinç, E. (2025) · Journal of Educational Technology Systems · doi:10.1177/00472395241312652 · Cited by 4 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Yang, S. C. (2026) · Computers in Human Behavior · doi:10.1016/j.chb.2026.108933 · Cited by 17 · see citing works
Wong, W. L., & Fong, S. (2025) · Applied Language Sciences · doi:10.65553/als.250105 · Cited by 2 · see citing works
Shadiev, R., Nguyen, T.-R.-G., & Hwang, W. (2026) · Interactive Learning Environments · doi:10.1080/10494820.2026.2677725 · Cited by 2 · see citing works
Theoretical Disruption: AI-Driven Language Systems and the Reformulation of Linguistics
Nuri, A. B., Seyidov, R., Hashimov, A., Анатольевна, К. Л., Ismayil, Z., Alizada, P., Tavberidze, N., & Ahmadov, H. (2026) · Journal of Ethnic and Cultural Studies · doi:10.66815/ejecs/3065
Institutional Readiness for Responsible AI
Oyenuga, M. O. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-9898-3.ch013
A critical discourse analysis of English distance-learning modules
Mardiana, R., Suryatiningsih, N., Lolowang, I. S., Wulandari, E., Wongkar, Y. H., & Utami, D. (2026) · Quarterly review of distance education · doi:10.1108/qrde-05-2026-0030
Generative AI-Driven Personalized English Writing Feedback
He, Y., & Xiao, H. (2026) · Information Resources Management Journal · doi:10.4018/irmj.424033
- Authors
- Sun, J., & Rui, J.
- Affiliations
- Jiangsu Ocean University (CN)
- Minjiang University (CN)
- Corresponding author
- Rui, Jingying
- ORCID
- Rui, Jingying
- Venue
- Journal of Educational Computing Research
- Publisher
- SAGE Publications
- Published
- 2025-11-17
- Volume
- 64
- Issue
- 3
- Pages
- 539-570
- ISSN
- 0735-6331 (ISSN-L)
- 1541-4140
- OpenAlex
- W4416291177
- Cited by
- 6
- References
- 38
- Citations by year
- 2026: 6
- FWCI
- 21.61
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Journal of Educational Computing Research (publishedVersion)
- Topics
- Social Sciences › Arts and Humanities › Language and Linguistics › EFL/ESL Teaching and Learning
- Physical Sciences › Computer Science › Computer Science Applications › Teaching and Learning Programming
- Physical Sciences › Computer Science › Artificial Intelligence › Intelligent Tutoring Systems and Adaptive Learning
- Keywords
- AI-assisted language learning, collaborative dialogue, computational thinking, language proficiency, EFL learners, conversation networks, learner control, technology dependence, sociocultural theory, Chinese learners
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
