Chatbot-mediated Learning: Conceptual Framework for the Design of Chatbot Use Cases in Education

“…theory (Deci & Ryan, 2012) Autonomy experience Social relatedness Competence experience Self-efficacy / social cognitive theory (Bandura, 1997) Self-Mastery; Self-Regulation Role-Modelling; Verbal persuasion Self-Efficacy; Feedback Self-regulation theory (Leventhal et al.,…”

“From an ultimate technological perspective, the goal of a chatbot and consequently the evaluation focus might be to pass the so-called Turing test (cf. Turing, 1950), meaning that the optimal chatbot cannot be distinguished from humans.”

“While the first chatbot named Eliza was already developed over 50 years ago by Weizenbaum (1966), the major developments have happened in recent years.”

“A corpus of learning-related theories (Bandura, 1997; Deci & Ryan, 2012; Leventhal et al., 1984) and reports (Bransford et al., 2000; Hattie & Yates, 2013; ISTE, 2017) was chosen based on broad acceptance in educational research and related based on its key concepts and constructs for learning…”

“From a business perspective chatbots mainly allow to improve customer service and to reduce service costs, from a user’s perspective important motivations to use chatbots are productivity, entertainment, social factors and novelty interaction (Adamopoulou & Moussiades, 2020).”

“And in terms of developing a chatbot for learning purposes, Smutny and Schreiberova (2020) offer a list of attributes describing the quality of an educational chatbot within the categories teaching (e.g. set goals and monitor learning progress), humanity (e.g. able to maintain themed discussion),…”

“…traditional intelligent tutoring systems or pedagogical agents in e-learning scenarios, chatbots do not only give instructions or provide feedback, but can also react to individual intents and create a real personalization and more importantly, a learner-centred approach (Winkler & Söllner, 2018).”

“While we refer to both categories as chatbots, other researchers with a more technical perspective prefer the umbrella term ‘conversational systems’ and then differentiate between more domain- or task-specific ‘dialog systems’ and generic ‘chatbots’ (cf. Chen et al., 2017).”

“A corpus of learning-related theories (Bandura, 1997; Deci & Ryan, 2012; Leventhal et al., 1984) and reports (Bransford et al., 2000; Hattie & Yates, 2013; ISTE, 2017) was chosen based on broad acceptance in educational research and related based on its key concepts and constructs for learning…”

“Fryer et al. (2019) summarize that in early research chatbots as language practice tools were shown to be useful for advanced and motivated students, but showed limitations in terms of in- and output quality.”

“Zanzotto (2019) calls for responsible AI with a "human-in-theloop" and a clear knowledge life cycle to prevent a bias of the AI or the chatbot.”

“…(ISTE, 2017) Communication skills; decision making Collaboration skills; creativity; empathy Critical thinking; Use of technology Visible learning (Hattie & Yates, 2013) Feedback; Self-Verbalisation Reciprocal teaching (dialog based) Formative evaluation; Meta-cognitive strategy Table 3:…”

“A future chatbot might also combine and integrate the intelligence of the different student-facing learning analytics systems distinguished by Bodily and Verbert (2017): Learning Analytics Dashboards, Educational Recommender Systems, Educational Data Mining Systems, Intelligent Tutoring Systems.”

“…Webservice (on any device) Social Media (known channel) Embedded in LMS (on any device) Content dimensions Contextual data integration Cf. (Molnar & Szuts, 2018; Winkler & Söllner, 2018) Basic domain knowledge database Basic context data (time, location, user) Personalisation with…”

“Since chatbot projects in education, just like innovations in learning technologies, are often driven by a technological direction, it is recommended to combine the pedagogical with the technological perspective at an early stage to develop a shared new vision of learning (Dillenbourg, 2016).”

“…• Analysis of real dialogs and questions • Creation of bot scripts • Bot training by defining intents • Bot skills development • Testing the chatbot • Optimize in productive use When designing chatbot dialogs, general and education specific design principles can be helpful (cf. Yu et al., 2016).”

“In addition to data protection, there is an important demand for explainable AI (XAI), which is particularly important in the field of education (Gunning, 2017).”

“Dimensions Characteristics (can be met supplementary) Pedagogical dimensions Chatbot role in individual learning setting Cf. (Garcia Brustenga et al., 2018) Support assistant e.g. research assistant, FAQ, Nerdybot Conversational partner e.g. communication trainer, tutor Recommender system e.g.…”

“This leads to complementary but different research procedures and evaluation approaches (Hobert, 2019).”

“Most studies, that have shown successful implementations of chatbot learning scenarios (cf. Dutta, 2017; Goel et al., 2016; Huang et al., 2017), are based on projects with isolated chatbot tasks, e.g. to answer frequently asked questions, to handle forum posts, or to ask questions in language…”

“Previous research on chatbots in education often focuses on designing messenger-like chatbots but there might be a lack on generalizable results (Meyer von Wolff et al., 2020).”

“Future chatbots might include more collaborative learning, analytics (cf. Ifenthaler & Schumacher, 2016) and metacognition functions, use more advanced AI and generative models, and additionally retrieve and process social and context information (maturity level 2-3).”

“Dimensions Characteristics (can be met supplementary) Pedagogical dimensions Chatbot role in individual learning setting Cf. (Garcia Brustenga et al., 2018) Support assistant e.g. research assistant, FAQ, Nerdybot Conversational partner e.g. communication trainer, tutor Recommender system e.g. learning path recommenda…”

“Dimensions Characteristics (can be met supplementary) Pedagogical dimensions Chatbot role in individual learning setting Cf. (Garcia Brustenga et al., 2018) Support assistant e.g. research assistant, FAQ, Nerdybot Conversational partner e.g. communication trainer, tutor Recommender system e.g. learning path recommenda…”

“…chatbots, including ChatGPT, have demonstrated the potential to reshape and revolutionize education, creating new, novel opportunities for enhancing the learning process and outcomes [83], facilitating different learning approaches, and offering a range of pedagogical benefits [19, 43, 72].”

“In addition, we address the need for better management tools for educators and the creation of personalised learning experiences, as highlighted by Hedderich et al. [32], Sonderegger and Seufert [33], and Gan et al. [34], emphasising the practical aspects of using AI in education.”

“Studies that build on AI-powered tools to support both personalisation and collaboration have the potential to enhance lifelong learning (Sonderegger and Seufert 2022).”

“In which areas and how this technology, which is based on technologies such as artificial intelligence, machine learning, natural language processing ([1], [2], [3] , [4], [5] , [6]), is used in education is an issue that needs to be examined.”

“8 According to Sonderegger and Seufert (2022), 8 there are three maturity levels of chatbots in education: chatbots for individual chatbot learning based on domain knowledge, chatbots for social or collaborative chatbot learning based on social and contextual knowledge, and chatbots for…”

“Sin embargo, se proyecta que podrán asumir el papel de profesor, tutor, asistente personal o analista de aprendizaje (Sonderegger & Seufert, 2022).”

“Study Amount IEEE [3], [4], [5], [6], [7] 5 Scopus [8], [9], [10], [11], [12] [13], [14], [15], [16], [17] [18], [19], [20], [21], [22] [23], [24], [25], [26], [27] 20 ACM [28], [29], [30], [31], [32] [33], [34], [35], [36], [37] [38] 11 Science Direct [39] 1”

“To explore how novice designers engage with traditional CUIs in a design feedback context, we conducted a formative study.”

“A N A BSOLUTE P RECONDITION : D IDACTIC E MBEDDING Initial attempts have been made to categorize standard LLM-based chatbots according to the roles they can take on in teaching and learning contexts [3], [11], [18].”

“(Sonderegger and Seufert 2022) propose a framework and address several limitations regarding data privacy and dependence on big technology suppliers.”

“Perceived security and trust not only affect credibility but also influence positive attitudes and adoption decisions [15].”