Chatbot Development Using LangChain: A Case Study to Foster Critical Thinking and Creativity
Open accessAbstract
Critical thinking and creativity are fundamental skills for engineers and computer scientists. The emergence of Large Language Models (LLMs) able to create chatbots that use natural language is an opportunity for educators to foster these skills. The well-known risk of generative AI for potential misinformation offers fertile ground to practice critical thinking. This paper describes a hands-on experience within a database course, where students had to develop a chatbot using the LangChain framework, and to evaluate it from different points of view. The students were free to choose the domain of their chatbot. The learning goal was twofold: on the one hand, to make them practice with state-of-the-art technologies, and on the other hand to stimulate critical analysis on their output. The paper discusses the students’ evaluation of the chatbots under several metrics, including document retrieval, syntax and grammar accuracy, semantic relevance and information reliability. Students’ assessments were also compared to the teachers’ ones, to gain an insight on the critical attitude of the students and to offer a ground for discussion. The experience was stimulating and appreciated by the students. The final results highlight that the majority of students successfully produced chatbot responses that were grammatically and syntactically correct, and that consistently extracted pertinent sections from documents, yielding semantically relevant outputs. Despite these achievements, a significant portion of students expressed reservations about the reliability of the chatbot’s responses to prompts, gaining awareness of LLMs’ capability to generate responses that make sense to humans but may be potentially misleading.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
A few studies used chatbot-specific evaluation rubrics, scoring on dimensions such as information accuracy, syntactic coherence, and semantic relevance.
“A few studies used chatbot-specific evaluation rubrics, scoring on dimensions such as information accuracy, syntactic coherence, and semantic relevance [89].”
From A Systematic Literature Review on the Pedagogical Implications and Impact of GenAI on Students' Critical Thinking (2026) · cites this paper as [89]
A few studies developed custom GPT-based systems.
“Other tools such as Gemini, Ernie Bot, and multi-tool workflows were less common [61,70], and a few studies developed custom GPT-based systems ( [50,59,84,89,90]).”
From A Systematic Literature Review on the Pedagogical Implications and Impact of GenAI on Students' Critical Thinking (2026) · cites this paper as [89]
Farinetti and Canale found enhanced evaluation, explanation skills and increased students' awareness of AI limitations and hallucinations.
“They found enhanced evaluation, explanation skills and increased students' awareness of AI limitations and hallucinations.”
From A Systematic Literature Review on the Pedagogical Implications and Impact of GenAI on Students' Critical Thinking (2026) · cites this paper as [89]
Cites
Thornhill-Miller, B., Camarda, A., Mercier, M., Burkhardt, J.-M., Morisseau, T., Bourgeois-Bougrine, S., Vinchon, F., El Hayek, S., Augereau-Landais, M., Mourey, F., et al. (2023) · Journal of Intelligence · doi:10.3390/jintelligence11030054 · Cited by 768 · see citing works
Fostering Students' Creativity and Critical Thinking
Vincent‐Lancrin, S., González-Sancho, C., Bouckaert, M., de Luca, F., Fernández-Barrera, M., Jacotin, G., Urgel, J., & Vidal, Q. (2019) · Educational research and innovation · doi:10.1787/62212c37-en · Cited by 402 · see citing works
How does generative artificial intelligence impact student creativity?
Habib, S., Vogel, T., Anli, X., & Thorne, E. (2024) · Journal of Creativity · doi:10.1016/j.yjoc.2023.100072 · Cited by 364 · see citing works
Chatbot: An Education Support System for Student
Clarizia, F., Colace, F., Lombardi, M., Pascale, F., & Santaniello, D. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01689-0_23 · Cited by 277 · see citing works
Artificial Intelligence & Creativity: A Manifesto for Collaboration
Vinchon, F., Lubart, T., Bartolotta, S., Gironnay, V., Botella, M., Bourgeois-Bougrine, S., Burkhardt, J., Bonnardel, N., Corazza, G. E., Glăveanu, V., et al. (2023) · The Journal of Creative Behavior · doi:10.1002/jocb.597 · Cited by 273 · see citing works
Chatgpt for next generation science learning
Zhai, X. (2023) · XRDS Crossroads The ACM Magazine for Students · doi:10.1145/3589649 · Cited by 178 · cited 1× · see citing works
“The use cases cover different educational challenges, ranging from creation of artifacts, evaluation and grading [5, 13, 22, 23].”
MethodHow ChatGPT Will Change Software Engineering Education
Daun, M., & Brings, J. (2023) · ACM Conference on Innovation and Technology in Computer Science Education (ITiCSE) · doi:10.1145/3587102.3588815 · Cited by 148 · see citing works
A Literature Review of Critical Thinking in Engineering Education
Ahern, A., Dominguez, C., McNally, C., O’Sullivan, J. J., & Pedrosa, D. (2019) · Studies in Higher Education · doi:10.1080/03075079.2019.1586325 · Cited by 127 · see citing works
AI-Assisted Learning with ChatGPT and Large Language Models: Implications for Higher Education
Laato, S., Morschheuser, B., Hamari, J., & Björne, J. (2023) · IEEE International Conference on Advanced Learning Technologies (ICALT) · doi:10.1109/icalt58122.2023.00072 · Cited by 103 · see citing works
Exploring the Role of ChatGPT in Education: Applications and Challenges
Mosaiyebzadeh, F., Pouriyeh, S., Parizi, R. M., Dehbozorgi, N., Dorodchi, M., & Batista, D. M. (2023) · Conference on Information Technology Education · doi:10.1145/3585059.3611445 · Cited by 80 · see citing works
Large Language Models for Education: Grading Open-Ended Questions Using ChatGPT
Pinto, G., Cardoso-Pereira, I., Monteiro, D., Lucena, D., Souza, A., & Gama, K. (2023) · Proceedings of the XXXVII Brazilian Symposium on Software Engineering · doi:10.1145/3613372.3614197 · Cited by 76 · see citing works
Chatbot Development as a Digital Learning Tool to Increase Students’ Research Knowledge
Vanichvasin, P. (2021) · International Education Studies · doi:10.5539/ies.v14n2p44 · Cited by 75 · see citing works
Exploring ChatGPT’s impact on post-secondary education: A qualitative study
Rajabi, P., Taghipour, P., Cukierman, D., & Doleck, T. (2023) · Proceedings of the 25th Western Canadian Conference on Computing Education · doi:10.1145/3593342.3593360 · Cited by 57 · see citing works
ChatGPT for Teaching and Learning: An Experience from Data Science Education
Zheng, Y. (2023) · Conference on Information Technology Education · doi:10.1145/3585059.3611431 · Cited by 56 · see citing works
"\"Call me Kiran\" – ChatGPT as a Tutoring Chatbot in a Computer Science Course"
Rajala, J., Hukkanen, J., Hartikainen, M., et al. (2023) · International Academic Mindtrek Conference (Mindtrek) · doi:10.1145/3616961.3616974 · Cited by 33 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Balart, T., Díaz, B., & Shryock, K. (2026) · Algorithms · doi:10.3390/a19030179 · Cited by 15 · see citing works
“Other tools such as Gemini, Ernie Bot, and multi-tool workflows were less common [61,70], and a few studies developed custom GPT-based systems ([50,59,84,89,90]).”
Influential
- Authors
- Farinetti, L., & Canale, L.
- Affiliations
- Politecnico di Torino (IT)
- Radiotelevisione Italiana (Italy) (IT)
- Venue
- Proceedings of the 2024 on Innovation and Technology in Computer Science Education V. 1
- Publisher
- ACM
- Published
- 2024-07-03
- Pages
- 401-407
- OpenAlex
- W4400268423
- Cited by
- 16
- References
- 15
- Citations by year
- 2026: 5 · 2025: 11
- FWCI
- 6.08
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- ACM Conference on Innovation and Technology in Computer Science Education (ITiCSE) (publishedVersion, cc-by-nc-nd)
- PORTO@Iris (submittedVersion, cc-by-nc-nd)
- Topics
- Physical Sciences › Computer Science › Information Systems › Artificial Intelligence in Education
- Social Sciences › Education › Problem and Project Based Learning
- Physical Sciences › Computer Science › Computer Science Applications › Teaching and Learning Programming
- Keywords
- chatbot development, LangChain, critical thinking, creativity, AI-assisted learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
