Looking Beyond the Hype: Understanding the Effects of AI on Learning

Probably not a substantial claim: may not pass S3 (load-bearing).

AI literacy is becoming an increasingly important transversal skill for learners.

“Additionally, AI literacy is becoming an increasingly important transversal skill for learners (Ng et al., 2021).”

• cites Ng et al., 2021Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

The current publication boom in AI-enhanced education partly reflects a tendency to overhype recent developments, driven by neglect of previous theoretical and empirical insights about instructional mechanisms and learning, the framing of arguments, and study methods.

“In this article, however, we argue that the current publication boom partly reflects a tendency to overhype recent developments, driven by a neglect of previous theoretical and empirical insights about instructional mechanisms and learning, the way arguments are framed, and the study methods used.”

May combine two claims.

LLMs provide outputs that can be misleading, biased, or incorrect.

“Especially AI systems that analyze and generate language using algorithms trained on large datasets, such as LLMs, provide outputs that can be misleading, biased, or incorrect.”

• cites Lee et al., 2024Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

Developing human knowledge and skills remains crucial for tasks requiring deep understanding, ethical considerations, and creative problem-solving, which cannot be entirely outsourced to AI.

“However, developing human knowledge and skills remains crucial, especially for tasks requiring deep understanding, ethical considerations, and creative problem-solving, which cannot be entirely outsourced to AI”

May combine two claims.

The probabilistic nature of LLMs implies they generate responses based on likelihood, not verified information, leading to plausible sounding but sometimes incorrect statements.

“In addition, the probabilistic nature of LLMs implies they generate responses based on likelihood, not verified information, leading to plausible sounding but sometimes incorrect statements.”

Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

Research on AI-enhanced learning should integrate theoretical, empirical, and methodological insights from prior research on the learning effects of digital technologies, including AIED.

“As this research evolves, it is crucial to integrate theoretical, empirical, and methodological insights from prior research on the learning effects of digital technologies, including AIED.”

Probably not a substantial claim: may not pass S4 (based).

With advancements in AI, especially generative AI, an increasing range of tasks can be outsourced to AI systems.

“With advancements in AI, especially generative AI, an increasing range of tasks can be outsourced to AI systems (e.g., many well-defined data analysis tasks), challenging educational systems and their targeted objectives.”

Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

AI holds significant potential for enhancing student learning.

“Artificial intelligence (AI) holds significant potential for enhancing student learning.”

Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

Research on AI-enhanced learning should prioritize cognitive learning processes and outcomes to maximize AI's potential for effective learning.

“This paper argues that research on AI-enhanced learning should prioritize cognitive learning processes and outcomes to maximize AI's potential for effective learning.”

Probably not a substantial claim: may not pass S4 (based).

New research trends on AI-enhanced learning depart from traditional AIED research, which focused on systematically establishing insights on instructional effectiveness with rigorous research methods.

“These new research trends depart from the traditional AIED research, which has focused on systematically establishing insights on instructional effectiveness with rigorous research methods.”

Probably not a substantial claim: may not pass S2 (checkable).

Effectively teaching knowledge and skills and preparing individuals for complex real-world problem-solving requires engaging them in relevant cognitive learning activities.

“Effectively teaching knowledge and skills and preparing individuals for complex real-world problem-solving requires engaging them in relevant cognitive learning activities.”

Probably not a substantial claim: may not pass S2 (checkable).

Similar outcomes could potentially occur with educational AI systems that either relieve students of essential learning processes or allow them to outsource critical learning processes.

“While the studies of Stadler et al. (2024) and Fan et al. (2024) investigate a non-educational generative AI system in educational settings, similar outcomes could potentially occur with educational AI systems that either relieve students of essential learning processes or allow them to outsource critical learning processes.”

• cites Stadler et al. (2024)Not checked yet

  Not checked in the free run.

• cites Fan et al. (2024)Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Hybrid intelligence approaches support continuous learning and upskilling, ensuring humans maintain and develop skills through human-AI collaboration.

“Such hybrid intelligence approaches were suggested to support continuous learning and upskilling, ensuring humans maintain and develop skills through human-AI collaboration (Järvelä et al., 2023;Rafner et al., 2021).”

• cites Järvelä et al., 2023Not checked yet

  Not checked in the free run.

• cites Rafner et al., 2021Not checked yet

  Not checked in the free run.

May combine two claims.

Over-reliance on algorithms may weaken decision-making and judgment skills, resulting in poor choices when AI support is unavailable.

“Over-reliance on algorithms may weaken decision-making and judgment skills, resulting in poor choices when AI support is unavailable (Sutton et al., 2018).”

• cites Sutton et al., 2018Not checked yet

  Not checked in the free run.

May combine two claims.

The ChatGPT group showed improved task performance during the supported intervention phase compared to other groups, but no differences in posttest knowledge gain or transfer.

“However, while the ChatGPT group, compared to the other groups, showed improved task performance during the supported intervention phase, there were no differences in the posttest knowledge gain or knowledge transfer.”

• cites Fan et al. (2024)Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S4 (based).

Current publications on AI for learning often focus primarily on the potential advantages of these technologies.

“While a balanced view on the potential of AI, particularly LLMs and generative AI, for learning is necessary, current publications often focus primarily on the potential advantages of these technologies for learning.”

May combine two claims.

Performance during an AI-supported intervention phase can suggest initial learning benefits but cannot be considered sufficient evidence for actual learning effects.

“However, although performance during a supported intervention phase can suggest initial learning benefits, it cannot be considered as sufficient evidence for actual learning effects.”

May combine two claims.

In Stadler et al. (2024), ChatGPT simplified task processing by reducing task-irrelevant extraneous load and load intrinsic to the learning task, but did not improve learning effectiveness and was accompanied by reduced cognitive processing depth and reduced cognitive learning outcomes.

“The findings indicated that ChatGPT simplified task processing by reducing task-irrelevant extraneous load and load that is intrinsic to the learning task (see Sweller 2010); however, this did not improve learning effectiveness, but was accompanied by reduced cognitive processing depth (germane load; see Sweller 2010) and reduced cognitive learning outcomes, as indicated by students' quality of”

• cites Stadler et al., 2024Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Lenient inclusion criteria in research syntheses can lead to a 'garbage in, garbage out' problem unless strict methodological and conceptual inclusion criteria are applied.

“This dynamic can easily lead to a “garbage in, garbage out” problem unless strict methodological and conceptual inclusion criteria are applied during study selection (Borenstein et al., 2021).”

• cites Borenstein et al., 2021 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S4 (based).

The effectiveness of AI is contingent upon the quality of the instruction provided.

“Nonetheless, it is crucial to be explicit about what conclusions can and cannot be drawn from each study to prevent overhyping the capabilities of the technology itself, considering that its effectiveness is contingent upon the quality of the instruction provided.”

Classified as probably not a substantial claim.

The ISAR model distinguishes four types of effects in AI-enhanced learning: inversion, substitution, augmentation, and redefinition.

“To offer an explicit terminology for the AI enhancement debate, classify possible implementations, and define different research directions, we suggest distinguishing four types of effects in the context of AI-enhanced learning: inversion effects, substitution effects, augmentation effects, and redefinition effects of AI in educational contexts, summarized in the ISAR model shown in Fig. 1.”

May combine two claims.

The ChatGPT group relied strongly on AI support and showed relatively low metacognitive processing compared to other support groups.

“A temporal process analysis of learners' metacognitive activities suggested that the ChatGPT group relied strongly on the AI support and showed relatively low metacognitive processing compared to the other support groups.”

• cites Fan et al. (2024)Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable).

To effectively enhance learning of knowledge and skills, especially transversal skills, educators need to initiate deep learning processes in students.

“To effectively enhance the learning of both knowledge and skills, especially transversal skills crucial for the twenty-first century, educators need to initiate deep learning processes in their students.”

May combine two claims.

Syntheses exclusively focusing on LLMs and generative AI usage are currently constrained by the limited number of available studies, which may lead some to apply lenient inclusion criteria.

“However, syntheses that exclusively focus on LLMs and generative AI usage are currently still rather constrained by the limited number of available studies, which may be the reason why some syntheses and meta-analyses with this specific focus apply rather lenient inclusion criteria.”

Probably not a substantial claim: may not pass S2 (checkable).

Without careful differentiation, meta-analyses risk an 'apples and oranges' issue, where studies with varying interventions and comparison conditions are mixed together to make broad claims about AI effectiveness.

“Without careful differentiation, meta-analyses risk an “apples and oranges” issue, where studies with varying interventions and comparison conditions are mixed together to make broad claims about AI effectiveness.”

Probably not a substantial claim: may not pass S4 (based).

Many recent primary studies and research syntheses have faced criticism for methodological issues that compromise the interpretability and validity of their findings.

“Indeed, many recent primary studies and research syntheses have faced criticism for methodological issues that compromise the interpretability and validity of their findings.”

Classified as probably not a substantial claim.

Substitution effects in AI-enhanced learning occur when AI provides instructional equivalence to non-AI alternatives without changing learners' cognitive processing depth and, therefore, without directly changing learning outcomes.

“Substitution effects in AI-enhanced learning occur when AI provides instructional equivalence to non-AI alternatives without changing learners' cognitive processing depth and, therefore, without directly changing learning outcomes.”

May combine two claims.

Learners may redirect spared resources toward more intensive practice or additional learning activities, though the extent depends on individual factors such as motivation and self-regulation.

“In such cases, learners may redirect their spared resources toward more intensive practice or additional learning activities, though the extent of such benefits likely depends on individual factors such as motivation and self-regulation.”

Probably not a substantial claim: may not pass S2 (checkable).

Augmentation effects occur when AI enhances instruction by providing additional cognitive learning support compared to a non-AI alternative.

“Augmentation effects occur when AI enhances instruction by providing additional cognitive learning support compared to a non-AI alternative.”

Probably not a substantial claim: may not pass S4 (based). May combine two claims.

Many studies fail to adequately acknowledge their limitations, thereby contributing to the hype about AI's effects on learning.

“Despite many examples of rigorous research, there are also many studies that fail to adequately acknowledge their limitations (a problem we will elaborate on below), thereby contributing to the hype about AI's effects on learning.”

Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

Technology in education has a dual role: instrumental, emphasizing that positive learning effects should be attributed to instruction rather than technology itself; and transformative, highlighting the potential to afford known and new possibilities for enhanced instruction.

“While these perspectives differ, together they underscore the dual role of technology in education: instrumental, emphasizing that any positive learning effects should be attributed to the instruction rather than the technology itself; and transformative, highlighting the potential to afford known and new possibilities for enhanced instruction.”

Classified as probably not a substantial claim.

Inversion effects in AI-enhanced learning occur when AI, intended to support deep learning, instead leads to reduced cognitive processing and learning outcomes, counteracting its intended benefits.

“Inversion effects in AI-enhanced learning occur when AI, intended to support deep learning, instead leads to reduced cognitive processing and learning outcomes, counteracting its intended benefits.”

May combine two claims.

Meta-analyses found medium to large learning benefits for ITS compared to self-reliant learning without support, and small to medium gains when comparing ITS to nonadaptive or less adaptive computer-assisted learning.

“For example, meta-analyses found medium to large learning benefits for ITS, which provide feedback and targeted hints, compared to self-reliant learning without support, and small to medium gains when comparing ITS to nonadaptive or less adaptive computer-assisted learning (Ma et al., 2014;Steenbergen-Hu & Cooper, 2014).”

• cites Ma et al., 2014Not checked yet

  Not checked in the free run.

• cites Steenbergen-Hu & Cooper, 2014 sourceNot checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

Redefinition effects occur when AI transforms learning tasks to foster deeper (constructive or interactive) learning, provided that the non-AI comparison condition does not already support deep learning processes.

“Redefinition effects occur when AI transforms learning tasks to foster deeper (constructive or interactive) learning, provided that the non-AI comparison condition does not already support deep learning processes.”

May combine two claims.

Scaffolding can adjust task difficulty and task sequencing, thereby creating personalized learning paths, which is often employed by ITS systems but also considered beneficial in other contexts, such as simulation-based learning.

“Additionally, scaffolding can adjust task difficulty and task sequencing, thereby creating personalized learning paths, which is a technique that is often employed by ITS systems but also considered beneficial in other contexts, such as simulation-based learning (Fischer et al., 2022;Holstein et al., 2018).”

• cites Fischer et al., 2022Not checked yet

  Not checked in the free run.

• cites Holstein et al., 2018Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Teachers can use a write-curate-verify approach for generating materials, by writing prompts, curating output, and verifying output, which keeps human in the loop to ensure high-quality results.

“Teachers can use a write-curate-verify approach for generating materials, by writing the prompts, curating the output, and verifying the output, which keeps the human in the loop to ensure high-quality results (Bai et al., 2024;Ninaus & Sailer, 2022).”

• cites Bai et al., 2024Not checked yet

  Not checked in the free run.

• cites Ninaus & Sailer, 2022Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Natural language processing combined with speech synthesis can analyze and generate spoken language, though this process may still experience time lags compared to text-based communication.

“Additionally, natural language processing techniques, combined with speech synthesis, can analyze and generate spoken language (e.g., in computer-generated animated agents; Fink et al., 2024), though this process may still experience time lags compared to text-based communication (Dekel et al., 2024).”

• cites Fink et al., 2024Not checked yet

  Not checked in the free run.

• cites Dekel et al., 2024Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

Key affordances of AI in education lie in implementing these instructional approaches more intensively than is currently done in many educational contexts.

“However, because key affordances of AI in education lie in implementing these instructional approaches more intensively than is currently done in many educational contexts, the following sections discuss these approaches from the perspectives of augmentation (AIenhanced cognitive support) and redefinition (AI-enhanced learning activities for deep learning).”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

Teacher oversight can be needed, keeping the human in the loop, to actively monitor AI-generated content and ensure that AI outputs remain accurate, relevant, and pedagogically sound in educational settings.

“Also, teacher oversight can be needed, keeping the human in the loop, to actively monitor AI-generated content and ensure that AI outputs remain accurate, relevant, and pedagogically sound in educational settings.”

Probably not a substantial claim: may not pass S4 (based).

Like students, teachers may also be at risk of overrelying on AI outputs and adopting them uncritically, particularly if they have limited awareness of AI's potential pitfalls.

“However, like students, teachers may also be at risk of overrelying on AI outputs and adopting them uncritically, particularly if they have limited awareness of AI's potential pitfalls.”

Probably not a substantial claim: may not pass S4 (based).

AI technologies can enhance cognitive support by providing additional feedback and scaffolding that optimize cognitive processing of various learning activities.

“Specifically, AI technologies can enhance cognitive support by providing additional feedback and scaffolding that optimize cognitive processing of various learning activities.”

Probably not a substantial claim: may not pass S4 (based). May combine two claims.

The validity and accuracy of AI-enhanced cognitive support are crucial for its effectiveness, as low-quality cognitive support can lead to learner disengagement, ultimately hindering the intended augmentation.

“The validity and accuracy of AI-enhanced cognitive support are crucial for its effectiveness, as low-quality cognitive support can lead to learner disengagement, ultimately hindering the intended augmentation.”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

Such issues might be mitigated by focusing on educational AI tools with built-in quality assurance mechanisms that ensure AI-generated materials adhere to course content.

“Such issues might be mitigated by focusing on educational AI tools with built-in quality assurance mechanisms that ensure, for example, that AIgenerated materials adhere to the course content.”

May combine two claims.

AI-based feedback can be complemented by AI-enhanced peer feedback as a scalable alternative to AI-enhanced teacher feedback; however, even with AI support, quality of peer feedback processes and outcomes may still vary.

“To keep the human in the loop and respond to concerns about dehumanizing learning through replacing human interaction, AIbased feedback can be complemented by AI-enhanced peer feedback as a scalable alternative to AI-enhanced teacher feedback; however, even with AI support, the quality of peer feedback processes and outcomes may still vary (Banihashem et al., 2024;Bauer et al., 2023).”

• cites Banihashem et al., 2024Not checked yet

  Not checked in the free run.

• cites Bauer et al., 2023 sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Log data analysis identifies student behavior patterns, enabling timely interventions.

“Log data analysis identifies student behavior patterns, enabling timely interventions (Lim et al., 2023).”

• cites Lim et al., 2023Not checked yet

  Not checked in the free run.

May combine two claims.

Effective feedback significantly impacts learning outcomes by offering insights into task performance, facilitating self-assessment, and guiding future efforts.

“Effective feedback significantly impacts learning outcomes by offering insights into task performance, facilitating self-assessment, and guiding future efforts.”

• cites Hattie & Timperley, 2007 sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Effectiveness of both teacher- and student-facing dashboards depends on dashboard usability and audience characteristics, as designing dashboards that provide accessible, relevant, and actionable information can be challenging, and users may struggle to translate insights into meaningful actions because of their knowledge and skills.

“However, the effectiveness of both teacher-and student-facing dashboards depends on the dashboard usability and audience characteristics, as designing dashboards that provide accessible, relevant, and actionable information can be challenging, and users may struggle to translate insights into meaningful actions because of their knowledge and skills (Jivet et al., 2017;Matcha et al., 2020).”

• cites Jivet et al., 2017Not checked yet

  Not checked in the free run.

• cites Matcha et al., 2020Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

While adequate prompting determines immediate output quality, fine-tuning with domain-specific data can further enhance alignment with domain knowledge, instructional accuracy, and educational needs.

“While adequate prompting determines the immediate output quality, fine-tuning with domainspecific data can further enhance alignment with domain knowledge, instructional accuracy, and educational needs.”

Probably not a substantial claim: may not pass S2 (checkable).

A straightforward option for generative AI to enhance deep learning activities in education is by assisting teachers with planning instructional approaches that afford deep learning processes.

“A straightforward option for generative AI to enhance deep learning activities in education is by assisting teachers with planning instructional approaches that afford deep learning processes.”

May combine two claims.

This offers advantages in accessibility and immediacy of answers while alleviating teacher workload, especially in educational contexts with many students.

“This offers advantages in the accessibility and immediacy of answers while alleviating teacher workload, especially in educational contexts with many students (e.g., lectures in higher education).”

Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

The most transformative potential of AI in education may lie in redefinition effects, where AI is used to redesign learning tasks in ways that encourage students to engage in deep rather than shallow learning processes.

“The most transformative potential of AI in education may lie in redefinition effects, where AI is used to redesign learning tasks in ways that encourage students to engage in deep (constructive or interactive) rather than shallow (passive or active) learning processes.”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

Specialized interfaces that enhance output explainability and promote teachers' critical reflection and revision of AI-generated content might help mitigate these issues.

“Specialized interfaces that enhance output explainability (e.g., through annotations) and promote teachers' critical reflection and revision of AI-generated content might help mitigate these issues.”

May combine two claims.

Instructional materials such as interactive simulations can be generated with the help of AI and allow students' knowledge construction through exploration of complex concepts.

“Instructional materials such as interactive simulations can be generated with the help of AI (Bewersdorff et al., 2025) and allow students' knowledge construction through the exploration of complex concepts (Kali & Linn, 2008).”

• cites Bewersdorff et al., 2025Not checked yet

  Not checked in the free run.

• cites Kali & Linn, 2008Not checked yet

  Not checked in the free run.

May combine two claims.

Rule-based systems deliver feedback and scaffolding based on predefined rules but lack flexibility, while machine learning and deep learning systems offer personalized feedback by analyzing complex data patterns.

“Rule-based systems deliver feedback and scaffolding based on predefined rules but lack flexibility, while machine learning and deep learning systems offer personalized feedback by analyzing complex data patterns (Sailer et al., 2024b;Zapata-Rivera & Arslan, 2024).”

• cites Sailer et al., 2024bNot checked yet

  Not checked in the free run.

• cites Zapata-Rivera & Arslan, 2024Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S4 (based).

In digital learning environments, AI can enhance the quality of feedback and scaffolding compared to nonadaptive or less adaptive technology-driven cognitive support.

“Additionally, in digital learning environments, AI can enhance the quality of feedback and scaffolding compared to nonadaptive or less adaptive technology-driven cognitive support.”

May combine two claims.

Different scaffolds are most effective for different learners; for example, worked examples may not be as beneficial to high-knowledge learners as reflection phases.

“Moreover, different scaffolds are most effective for different learners; for example, worked examples may not be as beneficial to high-knowledge learners as reflection phases (Chernikova et al., 2020).”

• cites Chernikova et al., 2020Not checked yet

  Not checked in the free run.

May combine two claims.

Generative AI creates personalized feedback and scaffolds based on interactions and performance data, enabling dynamic support systems; however, these systems may lack explainability and, if not designed in agreement with instructional principles, may also generate content that varies in accuracy, specificity, and pedagogical quality.

“Generative AI creates personalized feedback and scaffolds based on interactions and performance data, enabling dynamic support systems; however, these systems may lack explainability and, if not designed in agreement with instructional principles, may also generate content that varies in accuracy, specificity, and pedagogical quality (Banihashem et al., 2024).”

• cites Banihashem et al., 2024Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based). May combine two claims.

Using such constructive tasks may increase learner motivation, which could potentially help mitigate inversion effects.

“Using such constructive tasks may increase learner motivation, which could potentially help mitigate inversion effects.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Knowledge and skills related to digital technologies, including AI literacy, are crucial for raising awareness about problems such as biases in training data and outputs.

“Especially knowledge and skills related to digital technologies, including AI literacy, are crucial for raising awareness about problems such as biases in training data and outputs (Ng et al., 2021).”

• cites Ng et al., 2021Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Ruffle & Riley system employs AutoTutor's Expectation Misconception Tailoring as a tutoring model to structure dialogues and address student misconceptions.

“Similarly, the Ruffle & Riley system (Schmucker et al., 2024), a trialoguebased tutoring platform, employs AutoTutor's Expectation Misconception Tailoring as a tutoring model to structure dialogues and address student misconceptions.”

• cites Schmucker et al., 2024Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

In addition to a chat-based user interface, further key components include a learner model tracking students' knowledge and misconceptions, a domain model representing relevant subject information, and a tutoring model determining instructional strategies.

“However, in addition to a (chat-based) user interface, further key components include a learner model tracking students' knowledge and misconceptions, a domain model representing relevant subject information, and a tutoring model determining instructional strategies (D'Mello & Graesser, 2023).”

• cites D'Mello & Graesser, 2023Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

User-friendly, ethical, and regulation-compliant AI applications may facilitate the effective use of AI in education.

“Lastly, user-friendly, ethical, and regulation-compliant AI applications may facilitate the effective use of AI in education.”

May combine two claims.

Although it lacks a traditional domain model and learner model, limiting content quality assurance and tracking of learners' task mastery, the Socratic Playground for Learning uses a Socratic tutoring model to foster critical thinking and reflection by iteratively posing questions and guiding learners.

“Although it lacks a traditional domain model and learner model, limiting content quality assurance and tracking of learners' task mastery, the Socratic Playground for Learning uses a Socratic tutoring model to foster critical thinking and reflection by iteratively posing questions and guiding learners.”

• cites Zhang et al., 2024Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

These skills may influence the quality of interactions with AI systems, such as effective prompt-writing and assessing AI outputs.

“These skills may influence the quality of interactions with AI systems, such as effective prompt-writing and assessing AI outputs.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Recent ITS developments increasingly use LLMs for enhanced interactivity.

“Recent ITS developments increasingly use LLMs for enhanced interactivity.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Socratic Playground for Learning uses GPT-4 with assisted prompt engineering for multi-turn dialogues.

“For example, the Socratic Playground for Learning (Zhang et al., 2024) uses GPT-4 with assisted prompt engineering for multi-turn dialogues.”

• cites Zhang et al., 2024Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

Successful AI integration in education additionally requires that students and teachers have the necessary knowledge, skills, and motivation to interact effectively with AI systems.

“However, successful AI integration in education additionally requires that students and teachers have the necessary knowledge, skills, and motivation to interact effectively with AI systems.”

Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

Understanding the opportunities and limitations of AI-enhanced learning in any educational context requires a holistic perspective on the various conditions necessary for effective and sustainable AI integration in education.

“In conclusion, understanding the opportunities and limitations of AIenhanced learning in any educational context requires a holistic perspective on the various conditions necessary for effective and sustainable AI integration in education.”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

This access is vital to avoid amplifying the digital divide within and across countries.

“This access is vital to avoid amplifying the digital divide within and across countries (UNESCO, 2023).”

• cites UNESCO, 2023Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

A common form of conversational ITS involves dialogue-based interactions with a simulated tutor.

“A common form of conversational ITS involves dialogue-based interactions with a simulated tutor (e.g., AutoTutor; Graesser, 2016).”

• cites Graesser, 2016Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

In a trialogue, the student can be the tutor, explaining content to a simulated tutee with optional help from a simulated teacher.

“Alternatively, in a trialogue, the student can be the tutor, explaining content to a simulated tutee with optional help from a simulated teacher (Graesser et al., 2017).”

• cites Graesser et al., 2017Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Individual access to technology is essential for both teachers and students, ensuring they have the necessary devices and resources to engage with AI tools.

“Individual access to technology is essential for both teachers and students, ensuring they have the necessary devices and resources to engage with AI tools (Crompton, 2017).”

• cites Crompton, 2017Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing). May combine two claims.

Political and institutional regulations governing the ethical use of AI, data privacy, and security are necessary to protect all parties and may provide a structured approach to integrating AI in education.

“Political and institutional regulations governing the ethical use of AI, data privacy, and security are necessary to protect all parties and may provide a structured approach to integrating AI in education (Liu et al., 2020).”

• cites Liu et al., 2020Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Learning-by-design engages students in iterative cycles of designing, building, testing, and reflecting to deepen their learning.

“For example, learning-by-design engages students in iterative cycles of designing, building, testing, and reflecting to deepen their learning (Kolodner et al., 2003a).”

• cites Kolodner et al., 2003aNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

These examples illustrate how LLMs can be integrated with ITS components and pedagogical principles, but also highlight challenges in balancing flexibility, content quality, pedagogical quality, and personalized instruction.

“These examples illustrate how LLMs can be integrated with ITS components and pedagogical principles, but also highlight challenges in balancing flexibility, content quality, pedagogical quality, and personalized instruction.”

May combine two claims.

Constructive learning activities, where students actively generate formative outputs, foster deep understanding and skill development and can, in some cases, be enhanced by generative AI.

“Constructive learning activities, where students actively generate formative outputs, foster deep understanding and skill development and can, in some cases, be enhanced by generative AI.”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based). May combine two claims.

Robust digital infrastructure, equitable access to technology, and supportive policies are crucial for seamless implementation.

“Robust digital infrastructure, equitable access to technology, and supportive policies are crucial for seamless implementation.”

May combine two claims.

Instead of using a traditional domain model, Ruffle & Riley generates tutoring workflows from existing content and relies on chat logs and real-time responses rather than building a comprehensive learner model.

“Instead of using a traditional domain model, it generates tutoring workflows from existing content and relies on chat logs and real-time responses rather than building a comprehensive learner model.”

• cites Schmucker et al., 2024Not checked yet

  Not checked in the free run.

May combine two claims.

These factors are comparable to those influencing the integration of other digital technologies in education, but may partially be further specified for the context of AI.

“These factors are comparable to those influencing the integration of other digital technologies in education (Lachner et al., 2024;Sailer et al., 2021), which we will summarize below, but may partially be further specified for the context of AI.”

• cites Lachner et al., 2024Not checked yet

  Not checked in the free run.

• cites Sailer et al., 2021Not checked yet

  Not checked in the free run.

May combine two claims.

Research showed that technology-driven natural language interactions can foster deep conceptual learning by requiring explanation and reflection, and help develop essential communication skills.

“This research showed that technology-driven natural language interactions can foster deep conceptual learning by requiring explanation and reflection, and help develop essential communication skills (Rus et al., 2023).”

• cites Rus et al., 2023Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

Ongoing professional development may ensure that teachers remain updated with the latest AI advancements and are equipped to integrate these tools effectively into their teaching practices.

“Ongoing professional development may ensure that teachers remain updated with the latest AI advancements and are equipped to integrate these tools effectively into their teaching practices (Lindner et al., 2019;Williams et al., 2021).”

• cites Lindner et al., 2019Not checked yet

  Not checked in the free run.

• cites Williams et al., 2021Not checked yet

  Not checked in the free run.

May combine two claims.

Inclusive education can be facilitated through institutional access to technology, which is, however, also not a given.

“Inclusive education can be facilitated through institutional access to technology, which is, however, also not a given (Liu et al., 2024).”

• cites Liu et al., 2024Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

Natural language-based learning systems represent a powerful approach to realizing potential redefinition effects through AI-enhanced learning activities for deep learning.

“Nevertheless, natural language-based learning systems represent a powerful approach to realizing potential redefinition effects through AI-enhanced learning activities for deep learning.”

May combine two claims.

Flipped classroom approaches could be useful because constructive activities take place during class, which facilitates teacher interventions that guide students' use of AI, while passive knowledge transfer takes place outside of class time.

“Additionally, flipped classroom approaches could be useful because the constructive activities take place during class, which facilitates teacher interventions that guide students' use of AI, while passive knowledge transfer takes place outside of class time (Akçayır & Akçayır, 2018;Strelan et al., 2020).”

• cites Akçayır & Akçayır, 2018Not checked yet

  Not checked in the free run.

• cites Strelan et al., 2020Not checked yet

  Not checked in the free run.