The DOI matches its DataCite record.
Enhancing Critical Thinking in Education by means of a Socratic Chatbot
Open accessAbstract
While large language models (LLMs) are increasingly playing a pivotal role in education by providing instantaneous, adaptive responses, their potential to promote critical thinking remains understudied. In this paper, we fill such a gap and present an innovative educational chatbot designed to foster critical thinking through Socratic questioning. Unlike traditional intelligent tutoring systems, including educational chatbots, that tend to offer direct answers, the proposed Socratic tutor encourages students to explore various perspectives and engage in self-reflection by posing structured, thought-provoking questions. Our Socratic questioning is implemented by fine and prompt-tuning the open-source pretrained LLM with a specialized dataset that stimulates critical thinking and offers multiple viewpoints. In an effort to democratize access and to protect the students' privacy, the proposed tutor is based on small LLMs (Llama2 7B and 13B-parameter models) that are able to run locally on off-the-shelf hardware. We validate our approach in a battery of experiments consisting of interactions between a simulated student and the chatbot to evaluate its effectiveness in enhancing critical thinking skills. Results indicate that the Socratic tutor supports the development of reflection and critical thinking significantly better than standard chatbots. Our approach opens the door for improving educational outcomes by cultivating active learning and encouraging intellectual autonomy.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Emerging research has investigated the potential of integrating GenAI technologies to scale and automate Socratic pedagogical strategies.
“In response, emerging research has investigated the potential of integrating GenAI technologies as a robust opportunity to effectively scale and automate Socratic pedagogical strategies (Hung et al., 2024 ; Fakour & Imani, 2025 ; Favero et al., 2024 ).”
From Socratic Mind: Impact of a Novel GenAI-Powered Formative Assessment Tool on Student Learning and Higher-Order Thinking (2026) · cites this paper as Favero et al., 2024
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
No references could be resolved yet.
Research works printed but not linked: 33
- D. Henriksen, P. Mishra (Eds.), Creative Provocations: Speculations on the Future of Creativity, Technology & Learning, volume 7 of Creativity Theory and Action in Education, Springer International Publishing, Cham, 2022.
- M. A. Kuhail, N. Alturki, S. Alramlawi, K. Alhejori, Interacting with educational chatbots: A systematic review, Education and Information Technologies 28 (2023) 973-1018.
- Z. Liang, W. Yu, T. Rajpurohit, P. Clark, X. Zhang, A. Kaylan, Let GPT be a Math Tutor: Teaching Math Word Problem Solvers with Customized Exercise Gen- eration, 2023. ArXiv:2305.14386 [cs]. · arXiv
- J. Q. Pérez, T. Daradoumis, J. M. M. Puig, Rediscovering the use of chatbots in education: A systematic litera- ture review, Computer Applications in Engineering Education 28 (2020) 1549-1565.
- A. Kerlyl, P. Hall, S. Bull, Bringing chatbots into edu- cation: Towards natural language negotiation of open learner models, in: International conference on inno- vative techniques and applications of artificial intelli- gence, Springer, 2006, pp. 179-192.
- P. Smutny, P. Schreiberova, Chatbots for learning: A review of educational chatbots for the facebook mes- senger, Computers & Education 151 (2020) 103862.
- Y. Goda, M. Yamada, H. Matsukawa, K. Hata, S. Ya- sunami, Conversation with a chatbot before an on- line EFL group discussion and the effects on critical thinking, The Journal of Information and Systems in Education 13 (2014) 1-7.
- J. Rudolph, S. Tan, S. Tan, War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. the new AI gold rush and its impact on higher education, Journal of Applied Learning and Teaching 6 (2023).
- P. C. Abrami, R. M. Bernard, E. Borokhovski, D. I. Waddington, C. A. Wade, T. Persson, Strategies for teaching students to think critically: A meta-analysis, Review of educational research 85 (2015) 275-314.
- N. J. Alsaleh, Teaching critical thinking skills: Liter- ature review., Turkish Online Journal of Educational Technology-TOJET 19 (2020) 21-39.
- S. S. Barsoum, M. S. Elnagar, B. M. Awad, The ef- fectiveness of using a cognitive style-based chatbot in developing science concepts and critical thinking skills among preparatory school pupils, European Scientific Journal, ESJ (2022).
- B. H. Ang, S. D. Gollapalli, S.-K. Ng, Socratic Question Generation: A Novel Dataset, Models, and Evaluation, in: A. Vlachos, I. Augenstein (Eds.), Proceedings of the 17th Conference of the European Chapter of the Asso- ciation for Computational Linguistics, Association for Computational Linguistics, Dubrovnik, Croatia, 2023, pp. 147-165.
- D. Wood, J. S. Bruner, G. Ross, The role of tutoring in problem solving, Journal of child psychology and psychiatry 17 (1976) 89-100.
- L. S. Vygotsky, Thought and language, MIT press, 2012.
- P. Zare, J. Mukundan, The Use of Socratic Method as a Teaching/Learning Tool to Develop Students' Critical Thinking: a Review of Literature, Language in India 15 (2015).
- S. F. Dalim, A. S. Ishak, L. M. Hamzah, Promoting students' critical thinking through Socratic method: The views and challenges, Asian Journal of University Education 18 (2022) 1034-1047.
- A. P. Goldin, O. Pedroncini, M. Sigman, Producing or reproducing reasoning? Socratic dialog is very effec- tive, but only for a few, PLOS ONE 12 (2017) e0173584. doi:10.1371/journal.pone.0173584, publisher: Public Library of Science. · DOI
- K. Shridhar, J. Macina, M. El-Assady, T. Sinha, M. Ka- pur, M. Sachan, Automatic Generation of Socratic Sub- questions for Teaching Math Word Problems, 2022. ArXiv:2211.12835 [cs]. · arXiv
- E. Al-Hossami, R. Bunescu, J. Smith, R. Teehan, Can language models employ the Socratic method? exper- iments with code debugging, in: Proceedings of the 55th ACM Technical Symposium on Computer Science Education V. 1, 2024, pp. 53-59.
- K. Izumi, H. Tanaka, K. Shidara, H. Adachi, D. Kanayama, T. Kudo, S. Nakamura, Response gen- eration for cognitive behavioral therapy with large language models: Comparative study with Socratic questioning, arXiv preprint arXiv:2401.15966 (2024). · arXiv
- H. Touvron, L. Martin, K. Stone, P. Albert, A. Alma- hairi, Y. Babaei, N. Bashlykov, S. Batra, P. Bhargava, S. Bhosale, et al., Llama 2: Open foundation and fine- tuned chat models, arXiv preprint arXiv:2307.09288 (2023). · arXiv
- E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, W. Chen, LoRA: Low-rank adaptation of large language models, arXiv preprint arXiv:2106.09685 (2021). · arXiv
- T. Dettmers, A. Pagnoni, A. Holtzman, L. Zettlemoyer, QLoRA: Efficient finetuning of quantized LLMs, Ad- vances in Neural Information Processing Systems 36 (2024).
- K. Crowston, Amazon Mechanical Turk: A research tool for organizations and information systems schol- ars, in: Shaping the Future of ICT Research. Methods and Approaches: IFIP WG 8.2, Working Conference, Tampa, FL, USA, December 13-14, 2012. Proceedings, Springer, 2012, pp. 210-221.
- C. Hughes, Theory of knowledge aims, objectives and assessment criteria: An analysis of critical think- ing descriptors, Journal of Research in International Education 13 (2014) 30-45.
- Y. Abdullin, D. Molla-Aliod, B. Ofoghi, J. Yearwood, Q. Li, Synthetic dialogue dataset generation using LLM agents, arXiv preprint arXiv:2401.17461 (2024). · arXiv
- K. Papineni, S. Roukos, T. Ward, W. jing Zhu, BLEU: a method for automatic evaluation of machine transla- tion, 2002, pp. 311-318.
- C.-Y. Lin, F. J. Och, ORANGE: a method for evaluating automatic evaluation metrics for machine translation, in: COLING 2004: Proceedings of the 20th Interna- tional Conference on Computational Linguistics, COL- ING, Geneva, Switzerland, 2004, pp. 501-507.
- C.-Y. Lin, ROUGE: A package for automatic evaluation of summaries, in: Text Summarization Branches Out, Association for Computational Linguistics, Barcelona, Spain, 2004, pp. 74-81.
- S. Banerjee, A. Lavie, METEOR: An automatic metric for MT evaluation with improved correlation with hu- man judgments, in: Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Ma- chine Translation and/or Summarization, Association for Computational Linguistics, Ann Arbor, Michigan, 2005, pp. 65-72.
- L. Pan, W. Lei, T.-S. Chua, M.-Y. Kan, Recent ad- vances in neural question generation, arXiv preprint arXiv:1905.08949 (2019). · arXiv
- T. Zhang*, V. Kishore*, F. Wu*, K. Q. Weinberger, Y. Artzi, Bertscore: Evaluating text generation with bert, in: International Conference on Learning Repre- sentations, 2020.
- P. He, X. Liu, J. Gao, W. Chen, Deberta: Decoding- enhanced BERT with disentangled attention, in: In- ternational Conference on Learning Representations, 2021.
Cited by
Sorted by citations
Lee, J., Hung, J.-T., Yilmaz Soylu, M., Popescu, D., Cui, C. Z., Grigoryan, G., Joyner, D. A., & Harmon, S. W. (2026) · Technology, Knowledge and Learning · doi:10.1007/s10758-026-10007-6 · Cited by 23
Metadata
- Authors
- Favero, L., Pérez-Ortiz, J. A., Käser, T., & Oliver, N.
- Affiliations
- ELLIS Alicante (ES)
- University of Alicante (ES)
- École Polytechnique Fédérale de Lausanne (CH)
- Corresponding author
- Favero, Lucile
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2024
- Pages
- 17-32
- ISSN
- 1865-0929 (ISSN-L)
- arXiv
- 2409.05511
- OpenAlex
- W4412993375
- Cited by
- 119
- References
- 0
- Citations by year
- 2026: 77 · 2025: 40 · 2024: 2
- FWCI
- 42.52
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- Communications in computer and information science (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- Infoscience (Ecole Polytechnique Fédérale de Lausanne) (submittedVersion, cc-by-nc-nd)
- arXiv (Cornell University) (cc-by)
- Funders
- Generalitat Valenciana
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Intelligent Tutoring Systems and Adaptive Learning
- Social Sciences › Education › Artificial Intelligence in Education
- Social Sciences › Education › Education and Critical Thinking Development
- Keywords
- critical thinking, Socratic questioning, educational chatbots, large language models, prompt tuning, small language models, Llama 2, self-reflection, intellectual autonomy, active learning, multiple perspectives
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
