Climate-control ChatGPT for EFL writing and AI literacy
The DOI matches its Crossref record; 20 of the 20 references it cites were found in a registry or an index.
Abstract
ChatGPT is increasingly used in EFL writing, yet applied linguistics lacks robust methods for measuring whether human-AI dialogue improves learners’ AI literacy, harm-sensitive reasoning, and independent argumentative writing. This mixed-methods quasi-experimental study examines whether climate-control ChatGPT, a prompted interactional design that requires learners to justify, qualify, counterexample, and re-author their own claims, produces stronger gains than standard ChatGPT use and no-ChatGPT instruction. The study involved 120 undergraduate EFL learners in six intact second-year academic writing sections at a public university in Jordan. Participants were assigned to no-ChatGPT control, standard ChatGPT, or climate-control ChatGPT conditions. Data included no-AI diagnostic, pretest, posttest, and delayed transfer essays; ChatGPT dialogue logs; first and final drafts; learner reflections; uptake traces; stimulated-recall interviews; and 9,113 HARM-CAL annotated language units. HARM-CAL, Harm-sensitive Reasoning and Anthropomorphism Calibration in Language, classified learner language for naïve relativism, moral imposition, harm-sensitive reasoning, AI anthropomorphism, calibrated AI literacy, and reflective uncertainty. Results showed that the climate-control condition produced the largest gains in harm-sensitive AI literacy and argumentative writing quality, with advantages maintained on delayed no-AI transfer. It also reduced direct uptake, increased self-repair and authorial control, and strengthened calibrated distinctions between AI output competence and consciousness. The study contributes a classroom-tested intervention and a transparent computational framework that combines reproducible annotation procedures, a fully specified TF-IDF baseline, and archived transformer-family classifier outputs for analyzing responsible AI literacy in second-language argumentation.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Using thematic analysis in psychology
Braun, V., & Clarke, V. (2006) · Qualitative Research in Psychology · doi:10.1191/1478088706qp063oa · Cited by 194,414
S., N. L. (2017) · doi:10.1177/1609406917733847 · Cited by 14,051 · cited 1×
“The analysis was informed by the phased logic of thematic analysis and by qualitative trustworthiness criteria related to auditability, coherence, and analytic transparency (Braun and Clarke, 2006, 2021; Nowell et al., 2017).”
The emotional dog and its rational tail: A social intuitionist approach to moral judgment.
Haidt, J. (2001) · Psychological Review · doi:10.1037/0033-295x.108.4.814 · Cited by 8,048 · cited 1×
“Moral psychology has shown that moral judgment can involve emotional, intuitive, and deliberative processes (Greene et al., 2001; Haidt, 2001).”
ChatGPT for good? On opportunities and challenges of large language models for education
Kasneci, E., Sessler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., et al. (2023) · Learning and Individual Differences · doi:10.1016/j.lindif.2023.102274 · Cited by 7,326
Achieving integration in mixed methods designs-principles and practices
Fetters, M. D., Curry, L. A., & Creswell, J. W. (2013) · Health Services Research · doi:10.1111/1475-6773.12117 · Cited by 7,219 · cited 3×
“The explanatory mixed-methods structure allowed quantitative and computational patterns to be interpreted through qualitative evidence from learner reflections and stimulated-recall interviews (Fetters et al., 2013; Guetterman et al., 2015).”
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445922 · Cited by 6,988 · cited 3×
“This distinction is important because current debates about language models warn against confusing linguistic form with understanding (Bender and Koller, 2020; Bender et al., 2021).”
On seeing human: A three-factor theory of anthropomorphism
Epley, N., Waytz, A., & Cacioppo, J. T. (2007) · Psychological Review · doi:10.1037/0033-295x.114.4.864 · Cited by 4,069 · cited 4×
“Epley et al. (2007) show that humans readily attribute humanlike qualities to nonhuman agents under certain cognitive and social conditions.”
InfluentialModel Cards for Model Reporting
Mitchell, M., Wu, S., Zaldivar, A., Barnes, P., Vasserman, L., Hutchinson, B., Spitzer, E., Raji, I. D., & Gebru, T. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287596 · Cited by 2,681 · cited 4×
“…participant-level cross-validation split files, random seeds, preprocessing scripts, TF-IDF baseline hyperparameters, software and package versions, preserved transformer-family prediction outputs, and a short (Gebru et al., 2021; Mitchell et al., 2019; Papi et al., 2024; Storks et al., 2023).”
InfluentialClimbing towards NLU: On Meaning, Form, and Understanding in the Age of Data
Bender, E. M., & Koller, A. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.463 · Cited by 1,082 · cited 4×
“This distinction is important because current debates about language models warn against confusing linguistic form with understanding (Bender and Koller, 2020; Bender et al., 2021).”
InfluentialWhat is consciousness, and could machines have it?
Dehaene, S., Lau, H., & Kouider, S. (2017) · Science · doi:10.1126/science.aan8871 · Cited by 855 · cited 4×
“This focus is necessary because fluent language models can produce per-suasive moral language without understanding, consciousness, or suffering in the human sense (Bender and Koller, 2020; Bender et al., 2021; Dehaene et al., 2017).”
InfluentialPerkins, M. (2023) · Journal of University Teaching and Learning Practice · doi:10.53761/1.20.02.07 · Cited by 839
Ouyang, F., Zheng, L., & Jiao, P. (2022) · Education and Information Technologies · doi:10.1007/s10639-022-10925-9 · Cited by 831
Using ChatGPT for second language writing: pitfalls and potentials
Barrot, J. S. (2023) · Assessing Writing · doi:10.1016/j.asw.2023.100745 · Cited by 770 · cited 8×
“Much previous work has focused on final texts, feedback quality, learner perceptions, or writing scores (Barrot, 2023; Song and Song, 2023; Su et al., 2023; Yan, 2023).”
InfluentialSong, C., & Song, Y. (2023) · Frontiers in Psychology · doi:10.3389/fpsyg.2023.1260843 · Cited by 717
Impact of ChatGPT on learners in a L2 writing practicum: an exploratory investigation
Yan, D. (2023) · Education and Information Technologies · doi:10.1007/s10639-023-11742-4 · Cited by 713 · cited 8×
“…need frequent support but teachers cannot provide individualized feedback at every stage of composing (Alnemrat et al., 2025; Barrot, 2023; Kohnke et al., 2023; Song and Song, 2023; Yan, 2023; 10.3389/frai.2026.1869343 Frontiers in Artificial Intelligence 03 frontiersin.org Yang and Li, 2024).”
InfluentialTeacher support and student motivation to learn with Artificial Intelligence (AI) based chatbot
Chiu, T. K. F., Moorhouse, B. L., Chai, C. S., & Ismailov, M. (2024) · Interactive Learning Environments · doi:10.1080/10494820.2023.2172044 · Cited by 707 · cited 1×
“This pattern is consistent with research showing that teacher support is important for student motivation when learners use AI chatbots (Chiu et al., 2024) and with evidence that AI-mediated EFL learning can affect enjoyment, anxiety, and will-ingness to communicate (Zhang et al., 2024).”
Classroom dialogue: a systematic review across four decades of research
Howe, C., & Abedin, M. (2013) · Cambridge Journal of Education · doi:10.1080/0305764x.2013.786024 · Cited by 606
Linguistic Features of Writing Quality
McNamara, D. S., Crossley, S. A., & McCarthy, P. M. (2010) · Written Communication · doi:10.1177/0741088309351547 · Cited by 570 · cited 4×
“Writing development involves control over stance, cohesion, reasoning, register, and the linguistic expression of qualification (Aull and Lancaster, 2014; McNamara et al., 2010; Polio, 2017).”
InfluentialDigital support for academic writing: A review of technologies and pedagogies
Strobl, C., Ailhaud, E., Benetos, K., Devitt, A., Kruse, O., Proske, A., & Rapp, C. (2019) · Computers & Education · doi:10.1016/j.compedu.2018.12.005 · Cited by 298
Kazu, İ. Y., & Kuvvetli, M. (2023) · Education and Information Technologies · doi:10.1007/s10639-023-11756-y · Cited by 83
Research works printed but not linked: 29
- Alnemrat, 2025, AI vs. teacher feedback on EFL argumentative writing: a quantitative study, Front. Educ., 10, 1614673, 10.3389/feduc.2025.1614673 · DOI
- Blended learning with AI role-play in English for tourism and hospitality 103067 Aldamen H. Almashour M. Al-Deaibes M. Jarrah M. Alsharefeen R. Social Sciences & Humanities Open 14 · DOI
- Artstein, 2008, Survey article: inter-coder agreement for computational linguistics, Comput. Linguist., 34, 555, 10.1162/coli.07-034-R2 · DOI · Paper page
- Aull, 2014, Linguistic markers of stance in early and advanced academic writing: a corpus-based comparison, Written Commun., 31, 151, 10.1177/0741088314527055 · DOI · Paper page
- Bates, 2015, Fitting linear mixed-effects models using lme4, J. Stat. Softw., 67, 1, 10.18637/jss.v067.i01 · DOI · Paper page
- Braun, 2021, To saturate or not to saturate? Questioning data saturation as a useful concept for thematic analysis and sample-size rationales, Qual. Res. Sport, Exerc. Health, 13, 201, 10.1080/2159676X.2019.1704846 · DOI · Paper page
- Cohen, 1960, A coefficient of agreement for nominal scales, Educ. Psychol. Meas., 20, 37, 10.1177/001316446002000104 · DOI · Paper page
- D'Mello, 2014, Confusion can be beneficial for learning, Learn. Instr., 29, 153, 10.1016/j.learninstruc.2012.05.003 · DOI · Paper page
- Gebru, 2021, Datasheets for datasets, Commun. ACM, 64, 86, 10.1145/3458723 · DOI · Paper page
- Greene, 2001, An fMRI investigation of emotional engagement in moral judgment, Science, 293, 2105, 10.1126/science.1062872 · DOI · Paper page
- Guetterman, 2015, Integrating quantitative and qualitative results in health science mixed methods research through joint displays, Ann. Fam. Med., 13, 554, 10.1370/afm.1865 · DOI · Paper page
- Kohnke, 2023, ChatGPT for language teaching and learning, RELC J., 54, 537, 10.1177/00336882231162868 · DOI · Paper page
- Kyle, 2015, Automatically assessing lexical sophistication: indices, tools, findings, and application, TESOL Q., 49, 757, 10.1002/tesq.194 · DOI · Paper page
- Ladson-Billings, 1995, Toward a theory of culturally relevant pedagogy, Am. Educ. Res. J., 32, 465, 10.3102/00028312032003465 · DOI · Paper page
- Li, 2024, Evaluating the role of ChatGPT in enhancing EFL writing assessments in classroom settings: a preliminary investigation, Humanit. Soc. Sci. Commun., 11, 1268, 10.1057/s41599-024-03755-2 · DOI
- Lu, 2024, Can ChatGPT effectively complement teacher assessment of undergraduate students’ academic writing?, Assess. Eval. High. Educ., 49, 616, 10.1080/02602938.2024.2301722 · DOI · Paper page
- Mercer, 2012, Explaining the dialogic processes of teaching and learning: the value and potential of sociocultural theory, Learn. Cult. Soc. Interact., 1, 12, 10.1016/j.lcsi.2012.03.001 · DOI · Paper page
- Nakagawa, 2013, A general and simple method for obtaining R2 from generalized linear mixed-effects models, Methods Ecol. Evol., 4, 133, 10.1111/j.2041-210x.2012.00261.x · DOI · Paper page
- Papi, 2024, When good and reproducible results are a giant with feet of clay: the importance of software quality in NLP, Proc. 62nd Annu. Meet. Assoc. Comput. Linguist, 1, 3657, 10.18653/v1/2024.acl-long.200 · DOI
- Paris, 2012, Culturally sustaining pedagogy: a needed change in stance, terminology, and practice, Educ. Res., 41, 93, 10.3102/0013189X12441244 · DOI · Paper page
- Plonsky, 2014, How big is big? Interpreting effect sizes in L2 research, Lang. Learn., 64, 878, 10.1111/lang.12079 · DOI · Paper page
- Polio, 2017, Second language writing development: a research agenda, Lang. Teach., 50, 261, 10.1017/S0261444817000015 · DOI · Paper page
- Praphan, 2023, AI technologies in the ESL/EFL writing classroom: the villain or the champion?, J. Second. Lang. Writ., 62, 101072, 10.1016/j.jslw.2023.101072 · DOI
- Reimers, 2019, Sentence-BERT: sentence embeddings using Siamese BERT-networks, Proc. EMNLP-IJCNLP, 22, 3982, 10.18653/v1/D19-1410 · DOI · Paper page
- Samuel, 2023, Cultivation of human centered artificial intelligence: culturally adaptive thinking in education (CATE) for AI, Front. Artif. Intell., 6, 1198180, 10.3389/frai.2023.1198180 · DOI
- Storks, 2023, NLP reproducibility for all: understanding experiences of beginners, Proc. 61st Annu. Meet. Assoc. Comput. Linguist, 1, 10199, 10.18653/v1/2023.acl-long.568 · DOI
- Su, 2023, Collaborating with ChatGPT in argumentative writing classrooms, Assess. Writ., 57, 100752, 10.1016/j.asw.2023.100752 · DOI · Paper page
- Yang, 2024, ChatGPT for L2 learning: current status and implications, System, 124, 103351, 10.1016/j.system.2024.103351 · DOI
- Zhang, 2024, Artificial intelligence in EFL speaking: impact on enjoyment, anxiety, and willingness to communicate, System, 121, 103259, 10.1016/j.system.2024.103259 · DOI · Paper page
Metadata
- Authors
- Aldamen, H., Alawneh, S. M., Almashour, M., Al-Deaibes, M., & Alsharefeen, R.
- Venue
- Frontiers in Artificial Intelligence
- Publisher
- Frontiers Media SA
- Published
- 2026-07-22
- Volume
- 9
- ISSN
- 2624-8212
- Cited by
- 1
- References
- 20
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.