CoChat
Sign Up
Cited by 129Intelligent Tutoring Systems and Adaptive LearningArtificial Intelligence in EducationPsychological and Educational Research Studies

GPT-3-Driven Pedagogical Agents to Train Children’s Curious Question-Asking Skills

Open access
Abdelghani, Rania, Wang, Yen-Hsiang, Yuan, Xingdi, Wang, Tong, Lucas, Pauline, Sauzéon, Hélène, Oudeyer, Pierre-Yves·2024·International Journal of Artificial Intelligence in Education
DOI 10.1007/s40593-023-00340-7
Open this paper in CoChat — free account

Abstract

The ability of children to ask curiosity-driven questions is an important skill that helps improve their learning. For this reason, previous research has explored designing specific exercises to train this skill. Several of these studies relied on providing semantic and linguistic cues to train them to ask more of such questions (also called divergent questions ). But despite showing pedagogical efficiency, this method is still limited as it relies on generating the said cues by hand, which can be a very long and costly process. In this context, we propose to leverage advances in the natural language processing field (NLP) and investigate the efficiency of using a large language model (LLM) for automating the production of key parts of pedagogical content within a curious question-asking (QA) training. We study generating the said content using the "prompt-based" method that consists of explaining the task to the LLM in natural text. We evaluate the output using human experts annotations and comparisons with hand-generated content. Results suggested indeed the relevance and usefulness of this content. We then conduct a field study in primary school (75 children aged 9–10), where we evaluate children’s QA performance when having this training. We compare 3 types of content: 1) hand-generated content that proposes "closed" cues leading to predefined questions; 2) GPT-3-generated content that proposes the same type of cues; 3) GPT-3-generated content that proposes "open" cues leading to several possible questions. Children were assigned to either one of these groups. Based on human annotations of the questions generated, we see a similar QA performance between the two "closed" trainings (showing the scalability of the approach using GPT-3), and a better one for participants with the "open" training. These results suggest the efficiency of using LLMs to support children in generating more curious questions, using a natural language prompting approach that affords usability by teachers and other users not specialists of AI techniques. Furthermore, results also show that open-ended content may be more suitable for training curious question-asking skills.

AI snapshot

Written from the paper’s full text.

TL;DR

This study tested whether GPT-3 can automate curiosity-prompting conversational agents for children’s divergent question-asking, comparing hand-crafted and GPT-3-generated “closed” cues with GPT-3-generated “open” keyword cues. With 75 French 4th graders, the two closed agents produced similar question-asking performance, while the open GPT-3 agent led to significantly more divergent questions.

Key findings

  • Children using the open GPT-3 agent asked a significantly higher percentage of divergent questions than those using the hand-crafted incentive agent (t=-3.17, p=0.003) and the GPT-3 incentive agent (t=-1.88, p=0.04); the two incentive agents did not differ (t=0.88, p=0.38).
  • Children in the open condition used the agent’s cues more often (M=91.78%, SD=8.83) than children with the hand-crafted incentive agent (M=77.28%, SD=23.08; p=0.03) or the GPT-3 incentive agent (M=76.12%, SD=23.88; p=0.005).
  • GPT-3-generated cues were rated 100% not at all offensive and 100% relevant to the task; semantic relatedness did not differ across the three conditions (F(2,74)=0.41, p=0.66), and divergence level did not differ between the two incentive agents (t=0.9, p=0.37).
  • Across all conditions, divergent QA fluency improved pre- to post-intervention (time effect F(2,72)=46.95, p<0.001), and children’s perception of curiosity improved (time effect F(1,72)=15.74, p<0.0001), with self-efficacy increasing significantly only for the open GPT-3 group (p=0.014).

Methods

  • Three experimental conditions: hand-crafted incentive agent, GPT-3-driven incentive agent, and GPT-3-driven open agent; 75 4th-grade students aged 9–10.5 were assigned to groups of 24, 26, and 25, respectively.
  • The study ran over three 1-hour sessions: profile/pre-test, divergent-QA training with three texts and six questions per text (18 total), and post-test; measures included divergent QA fluency, curiosity trait/perception, motivation, cognitive load, and question quality.
  • GPT-3 (text-davinci-002, temperature 0.7) was used in a zero-shot prompt-based setup to generate cues offline; cues were checked, translated into French with DeepL, and stored in a database rather than exposing children directly to the model.

Limitations

  • The paper states that the system gave no feedback on children’s questions, so it is unclear whether improved self-efficacy came from performance perception or other factors such as exercise repetition or self-report bias.
  • The paper states that the training was short—one session—which may have inflated engagement and positive results; it also calls for more participants and more varied educational resources, including non-text-based materials.
  • Evident from the text: baseline device-use frequency differed across groups (p=0.008), though the authors report ANCOVA showing it did not affect the equilibrium (p=0.23); the study also used a pseudo-randomized assignment and a relatively small sample.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1007/s40593-023-00340-7

Cites

52 of 52 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

0 of 64 cited sources checked · 0 supported · 0 flagged

  • Findingown data
    § 7.4.2 On the perception of curiosity · p. 27

    The interaction between time and groups was significant for the Self-Efficiency sub-scale.

    “As shown the sub-figure (b) in Figure 13, the three groups had better scores for the post-intervention test and the interaction between time and the groups was significant (F('2,72)=0.038, p-value=0.04).”
    Quote in the paperSubstantial claim
  • Findingown data
    § 7.2.2 Relationship with the curiosity trait as reported by parents · p. 25

    The open agent led to a significantly different relationship between curiosity trait scores and divergent QA performance compared to the automated incentive agent (z=2.8, p=0.002).

    “the open agent led to a significantly different relationship (z=1.7, p-value=0.04) between the hand-crafted incentive agent and the automated open one and (z=2.8; p-value=0.002) between the automated incentive agent and the automated open one.”
    Quote in the paperSubstantial claim
  • Findingown data
    § 7.1.2 Generating the linguistic cues · p. 21

    The variety of question words generated by GPT-3 did not differ significantly from those generated by hand (Levene's test p=0.65).

    “Results showed no support to reject similarity between the two conditions (p-value= 0.65).”
    Quote in the paperNot a substantial claim
  • Findingreasoning
    § 8 Discussion · p. 28

    The system was more efficient in prompting children's curious question-asking when using an open agent that proposed keywords leading to several possible questions, rather than incentive agents that led to one specific and pre-defined question.

    “Moving forward, we saw that our system was more efficient in prompting children's curious question-asking when we had an open agent that proposed keywords leading to several possible questions, rather than the incentive agents that led to one specific and pre-defined question.”
    Quote in the paperSubstantial claim
  • Findingown data
    § 7.1.3 Generating the semantic cues · p. 22

    There was no significant difference between the three conditions (hand-crafted, GPT-3 incentive, GPT-3 open) in the semantic relatedness of the cues to the educational context (one-way ANOVA F(2,74)=0.41, p=0.66, power=0.98).

    “we see no significant difference between the three conditions for this measure. The one-way ANOVA test failed to exclude similarities between at least two groups (F(2,74)=0.41, p-value = 0.66), with a power test of 0.98”
    Quote in the paperNot a substantial claim
  • May combine two claims. 25
  • The quote only partly states this claim. 3
  • Classified as probably not a substantial claim. 28

  • May combine two claims. The quote only partly states this claim.

    Findingown data
    § Abstract · p. 2

    In the study with 75 children aged 9-10, question-asking performance was similar between children who interacted with the two 'closed' agents (hand-crafted and GPT-3-driven), but significantly better for participants who interacted with the GPT-3-driven 'open' agent.

    “Results showed a similar question-asking performance between children who had the two 'closed' agents, but a significantly better one for participants with the 'open' agent.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 8 Discussion · p. 28

    Children extracted the important ideas from the agent's semantic cues and used them to formulate a question on their own.

    “Instead, they extract the important ideas and used them to formulate a question on their own.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data
    § 3.3.1 Generating the agents' cues using chain-of-thoughts prompting in GPT-3 · p. 12

    There was no significant difference between story-then-instruction and instruction-then-story prompt variants.

    “We do not observe significant difference from these variants”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data
    § 8 Discussion · p. 28

    Adding an automatic translation step into French still led to encouraging general results regarding the quality of the prompts generated and their positive impacts on children's divergent question-asking performances.

    “Adding this translation layer still led us to rather encouraging general results wrt the quality of the prompts generated and their positive impacts on children's divergent question-asking performances.”
    Quote in the paperSubstantial claim

  • The quote only partly states this claim.

    Interpretationown data
    § Abstract · p. 2

    Using GPT-3 to automate the generation of curiosity-prompting cues is a valid way to facilitate the implementation of curiosity-stimulating learning technologies.

    “Our first results suggest the validity of using GPT-3 to facilitate the implementation of curiosity-stimulating learning technologies.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Generating the curiosity-prompting cues by hand for each educational resource is a very long and costly process, which makes the generalization and scale-up of the approach difficult.

  • May combine two claims.

    Premisecitation
    § 2.3 GPT-3 and prompt-based learning · p. 5

    Fine-tuning a pre-trained LLM can be problematic because it can be extremely costly and hard for super large models, data can be scarce, and the model is fine-tuned separately for each different task.

    “This method can be problematic for several reasons: it can be extremely costly and hard to perform for super large models that have an enormous number of parameters [44], data can be scarce in many cases, the model is fine-tuned separately for each different task, etc.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [44] Min, S., Lyu, X., Holtzman, A., Artetxe, M., Lewis, M., Hajishirzi, H., and Zettlemoyer, L. (2022). Rethinking the role of demonstrations: What makes in-context learning work? arXiv preprint arXiv:2202.12837 source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § 3.2.1 Choice of the interaction workflow · p. 10

    Giving children the choice over their tasks has a strong positive effect on their motivation and engagement, on the amount of knowledge they can learn, and on their perceived competence.

    “giving children the choice over their tasks has shown to have a strong positive effect not only on their motivation and engagement but also on the amount of knowledge they can learn and their perceived competence [10].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [10] Ceha, J., Chhibber, N., Goh, J., McDonald, C., Oudeyer, P. Y., Kulić, D., & Law, E. (2019). Expression of curiosity in social robots: Design, perception, and effects on behaviour. In Proceedings of the 2019 CHI conference on human factors in computing systems (pp. 1–12). source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § 1 Introduction · p. 3

    Curiosity-based question-asking behaviors are almost absent from today's classrooms, and children's inquiries are very often low-level, memory-based questions that do not require much cognitive work.

    “curiosity-based question-asking (QA) behaviors are shown to be almost absent from today's classrooms: children's inquiries are very often low-level questions that are memorybased and that do not require much cognitive work [16, 20]”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [16] Gordon, G., Breazeal, C., & Engel, S. (2015, March). Can children catch curiosity from a social robot?. In Proceedings of the tenth annual ACM/IEEE international conference on human-robot interaction (pp. 91–98). source

      Not checked in the free run.

    • cites [20] Guay, 2022, Applying self-determination theory to education: Regulations types, psychological needs, and autonomy supporting behaviors, Canadian Journal of School Psychology source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § 1 Introduction · p. 3

    Previous studies using conversational agents to stimulate children's questioning and exploration showed positive effects on children's curiosity-driven behaviors and on their domain-knowledge learning progress.

    “Their results showed positive effects on children's curiosity-driven behaviors as well as on their domain-knowledge learning progress, thus suggesting the pedagogical efficiency of their methods.”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [2] Alaimi, M., Law, E., Pantasdo, K. D., Oudeyer, P. Y., & Sauzéon, H. (2020). Pedagogical agents for fostering question-asking skills in children. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (pp. 1–13). New York, NY: Association for Computing Machinery. source

      Not checked in the free run.

    • cites [1] Abdelghani, 2022, Conversational agents for training curiosity-driven learning in children, International Journal of Human-Computer Studies source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation
    § 2.3 GPT-3 and prompt-based learning · p. 5

    Adapter-tuning methods add only a few trainable parameters per new task, allowing a high degree of parameter sharing.

    “We also find adapter-tuning methods [19] that only add a few trainable parameters per new task, allowing a high degree of parameter sharing.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [19] Graesser, A. C., Person, N., & Huber, J. (1992). Mechanisms that generate questions. Questions and Information Systems, 2, 167–187.

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § 2.1 Epistemic curiosity and QA training in education · p. 4

    Curiosity plays an important role in learning from infancy to adulthood, and in fostering academic achievement, particularly for children with low socio-economic status.

    “Curiosity plays an important role in learning from infancy [53] to adulthood [45], and in fostering academic achievement, particularly for children with low socio-economic status [50].”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites [53] Scardamalia, 1992, Text-based and knowledge based questioning by children, Cognition and Instruction source

      Not checked in the free run.

    • cites [45] Murayama, 2019, Process account of curiosity and interest: A reward-learning perspective, Educational Psychology Review source

      Not checked in the free run.

    • cites [50] Roebers, 2007, Favourable and unfavourable conditions for children’s confidence judgments, British Journal of Developmental Psychology source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation
    § 2.3 GPT-3 and prompt-based learning · p. 5

    Pre-trained large language models demonstrate impressive powers in generating natural language and are easier to use in downstream tasks.

    “the NLP field has been focusing on the use of pre-trained large language models (LLMs) as they demonstrate impressive powers in generating natural language and are easier to use in downstream tasks ([11, 7]).”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [11] Cordova, 1996, Intrinsic motivation and the process of learning: Beneficial effects of contextualization, personalization, and choice, Journal of Educational Psychology source

      Not checked in the free run.

    • cites [7] Blodgett, S. L., Lopez, G., Olteanu, A., Sim, R., & Wallach, H. (2021). Stereotyping norwegian salmon: An inventory of pitfalls in fairness benchmark datasets. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) (pp. 1004–1015). source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § 3 Study design · p. 7

    Giving children the possibility to skip a question allows them to report low certainty, which in turn can motivate curiosity-driven behaviors.

    “work in [46] suggests that giving children the possibility to skip a question allows them to report low certainty which, in its turn, can motivate curiosity-driven behaviors.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [46] Oudeyer, 2007, Intrinsic motivation systems for autonomous mental development, IEEE Transactions on Evolutionary Computation source

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Premisecitation
    § 1 Introduction · p. 3

    According to self-determination theory, competence, autonomy and relatedness are fundamental factors to promote for more efficient educational trainings.

    “following the self-determination theory [48, 17], these are fundamental factors to promote for more efficient educational trainings.”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [48] Raffel, 2020, Exploring the limits of transfer learning with a unified text-to-text transformer, Journal of Machine Learning Research

      Not checked in the free run.

    • cites [17] Goupil, 2023, Curiosity as a metacognitive feeling, Cognition source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 2.1 Epistemic curiosity and QA training in education · p. 4

    The more curious students are by trait, the more divergent-thinking questions they will ask.

    “research such as in [2] suggest that the more curious students are by trait, the more divergent-thinking questions they will ask”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [2] Alaimi, M., Law, E., Pantasdo, K. D., Oudeyer, P. Y., & Sauzéon, H. (2020). Pedagogical agents for fostering question-asking skills in children. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (pp. 1–13). New York, NY: Association for Computing Machinery. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation
    § 1 Introduction · p. 3

    Children's low-level questioning may be explained by their tendency to overestimate their knowledge, their incapacity to formulate syntactically-correct questions, and their fear of classmates' judgments.

    “Possible explanations of this phenomena can be children's tendency to overestimate their knowledge and, thus, not having the need to ask further questions [16], their incapacity to formulate syntactically-correct questions [20], their fear of their classmates' judgments [43], etc.”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [16] Gordon, G., Breazeal, C., & Engel, S. (2015, March). Can children catch curiosity from a social robot?. In Proceedings of the tenth annual ACM/IEEE international conference on human-robot interaction (pp. 91–98). source

      Not checked in the free run.

    • cites [20] Guay, 2022, Applying self-determination theory to education: Regulations types, psychological needs, and autonomy supporting behaviors, Canadian Journal of School Psychology source

      Not checked in the free run.

    • cites [43] Metcalfe, 2020, Epistemic curiosity and the region of proximal learning, Current Opinion in Behavioral Sciences source

      Not checked in the free run.

  • May combine two claims.

    Premisereasoning
    § 2.3 GPT-3 and prompt-based learning · p. 5

    Prefix-tuning and adapter-tuning have achieved comparable results to fine-tuning but still require machine learning expertise for their implementation, which can be a major brake for using them in educational settings.

    “These simpler techniques have achieved comparable results to fine-tuning but still require machine learning expertise for their implementation, which can be a major brake for using them in educational settings where most stakeholders do not have this expertise.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 2.2 Natural language processing and conversational agents for building educational applications · p. 5

    Earlier research has shown several positive pedagogical impacts of using NLP-based social conversational agents on children's perception of their own competencies and their attitudes towards learning.

    “earlier research has showed several positive pedagogical impacts of using NLP-based social conversational agents on children's perception of their own competencies and their attitudes towards learning ([51]).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [51] Rubin, O., Herzig, J., and Berant, J. (2021) Learning to retrieve prompts for in-context learning. arXiv preprint arXiv:2112.08633 source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 2.1 Epistemic curiosity and QA training in education · p. 4

    The precision of the questions children generated was a significant predictor for their curiosity, as reported by their teachers.

    “studies such as [58] presented a question-asking based exploration game and showed that the precision of the questions that children generated was a significant predictor for their curiosity, as reported by their teachers.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [58] Sultan, M. A., Bethard, S., & Sumner, T. (2014). Towards automatic identification of core concepts in educational resources. In IEEE/ACM Joint Conference on Digital Libraries (pp. 379–388). IEEE. source

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Premisecitation
    § 3.2.2 Choice of the cues proposed by the agent · p. 11

    Teachers' or researchers' choice of semantic cues is heavily influenced by their preconceptions and beliefs about children's knowledge gaps.

    “We also think that it is important to give different choices in order to avoid the cases where teachers' or researchers' choice of the semantic cues is heavily influenced by their preconceptions and beliefs about children's knowledge gaps [25, 59].”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [25] Jiang, 2021, How can we know when language models know? on the calibration of language models for question answering, Transactions of the Association for Computational Linguistics source

      Not checked in the free run.

    • cites [59] Ten, 2021, Humans monitor learning progress in curiosity-driven exploration, Nature Communications source

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Premisecitation
    § 1 Introduction · p. 3

    Large language models have shown an impressive ability in predicting word sequences that resemble the way people write.

    “the large language models (LLMs) that showed an impressive ability in predicting word sequences that resemble the way people write( [13], [61]...).”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [13] Diakidoy, 2002, Domain specificity in creativity testing: A comparison of performance on a general divergent-thinking test and a parallel, content-specific test, The Journal of Creative Behavior source

      Not checked in the free run.

    • cites [61] Vallerand, 1989, Construction et validation de l’échelle de motivation en éducation (EME), Canadian Journal of Behavioural Science / Revue Canadienne Des Sciences Du Comportement

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § 3.2.2 Choice of the cues proposed by the agent · p. 10

    Learners decide whether or not to engage in pursuing a piece of information based on whether they perceive it to be in their zone of proximal development, which differs among learners depending on their expertise for the subject matter.

    “learners decide whether or not to engage in pursuing a piece of information based on if they perceive that this latter will be in their ZPD or not; which differs among learners depending on their expertise for the subject matter at hand [40].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [40] Lu, Y., Bartolo, M., Moore, A., Riedel, S., & Stenetorp, P. (2021). Fantastically ordered prompts and where to find them: Overcoming few-shot prompt order sensitivity. arXiv preprint arXiv:2104.08786. source

      Not checked in the free run.

  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation
    § 2.1 Epistemic curiosity and QA training in education · p. 4

    Curiosity can be seen as a personality trait, a psycho-emotional state aroused by external situations, or a malleable skill that can be trained through teaching specific practices.

    “It can be seen as a a personality trait ([28]), a psycho-emotional state aroused by external situations([39], [27]),[4],...) or as a malleable skill that can be trained through teaching specific practices.”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites [28] Jost, 1998, Social metacognition: An expansionist review, Personality and Social Psychology Review source

      Not checked in the free run.

    • cites [39] Loewenstein, 1994, The Psychology of Curiosity: A Review and Reinterpretation, Psychological Bulletin source

      Not checked in the free run.

    • cites [27] Jones, 2018, “i know that now, I’m going to learn this next’’ promoting self-regulated learning with a robotic tutor, International Journal of Social Robotics source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 2.2 Natural language processing and conversational agents for building educational applications · p. 4

    NLP-based methods can be used to extract the key scientific ideas and concepts from science educational resources that human experts consider important for a better and deeper understanding of the domain.

    “authors in [56] for instance, showed that NLP-based methods can be used to extract the key scientific ideas and concepts from science educational resources that, according to human experts, are important for a better and deeper understanding of the domain.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [56] Steuer, T., Filighera, A., Meuser, T., & Rensing, C. (2021). I do not understand what i cannot define: Automatic question generation with pedagogically-driven content selection. arXiv preprint arXiv:2110.04123.

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 2.3 GPT-3 and prompt-based learning · p. 5

    Prefix-tuning updates only a small percentage of the model's parameters for task adaptation.

    “methods such as prefix-tuning have emerged [33], where only a small percentage of the model's parameters are updated for taskadaptation.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [33] Law, E., Yin, M., Goh, J., Chen, K., Terry, M. A., & Gajos, K. Z. (2016, May). Curiosity killed the cat, but makes crowd work better. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (pp. 4098–4110). source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § 3.2.2 Choice of the cues proposed by the agent · p. 10

    Divergent questions require higher-level thinking and are more related to epistemic curiosity.

    “we focus on divergent questions in general as they require higher-level thinking and are more related to epistemic curiosity ([14, 49]).”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [14] Gallagher, 1963, A preliminary report on analyses of classroom interaction, Merrill-Palmer Quarterly

      Not checked in the free run.

    • cites [49] Reio, 2000, Field investigation of the relationship among adult curiosity, workplace learning, and job performance, Human Resource Development Quarterly source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § 1 Introduction · p. 3

    Curiosity is a malleable skill that can be elicited by both verbal and non-verbal cues, and by promoting comfort with uncertainty and encouraging questioning to resolve it.

    “research shows strong evidence that curiosity is a malleable skill that can be elicited by both verbal and non-verbal cues [9, 15, 31], and by promoting comfort with uncertainty and encouraging questioning to resolve it [23].”
    Quote in the paperSubstantial claim0 of 4 sources checked
    • cites [9] Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., … & Amodei, D. (2020). Language models are few-shot learners. Advances in neural information processing systems, 33, 1877–1901.

      Not checked in the free run.

    • cites [15] Gall, 1970, The use of questions in teaching, Review of Educational Research source

      Not checked in the free run.

    • cites [31] Kasneci, 2023, Chatgpt for good? on opportunities and challenges of large language models for education, Learning and Individual Differences source

      Not checked in the free run.

    • cites [23] Humphries, 2015, Beyond who, what, where, when, why, and how: Preparing students to generate questions in the age of common core standards, Journal ofResearch in Childhood Education source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation
    § 2.2 Natural language processing and conversational agents for building educational applications · p. 4

    NLP-based methods have been shown to be efficient in question-generation tasks, which can help teachers alleviate the time-consuming process of creating assessment and evaluation materials.

    “NLP-based methods have also shown to be efficient in the question-generation tasks (i.e. generating questions from educational resources) which can help teachers alleviate the time-consuming process of creating the assessment and evaluation materials ([6], [54], ..).”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [6] Bjork, 2017

      Not checked in the free run.

    • cites [54] Silvervarg Flycht-Eriksson, A. S., and Jönsson, A. (2010). Towards a conversational pedagogical agent capable of affecting attitudes and self-efficacy. In Proceedings of the Second Workshop on Natural Language Processing in Support of Learning: Metrics, Feedback and Connectivity

      Not checked in the free run.

  • May combine two claims.

    Interpretationown data
    § 7.2.1 Divergent question-asking performance · p. 24

    The GPT-3-driven incentive agent, compared to the human-driven one, led to similar performances in quantity and quality of questions children generated, as well as similar dependency on semantic cues.

    “the GPT-3-driven incentive agent, compared to the human-driven one, led to similar performances in terms of quantity and quality of the questions children generated, as well as a similar dependency on the semantic cues.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Premisenone given
    § 3.2.3 Choice of the technology · p. 11

    GPT-3 has significantly more parameters and was trained on a text corpus containing a wider variety of topics compared to its predecessor GPT-2.

    “we choose to work with GPT-3 given that it has significantly more parameters and was trained upon text corpus containing a wider variety of topics, compared to its predecessor GPT-2 for example.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationreasoning
    § 9 Limitations and future directions · p. 30

    The short duration of the training may have facilitated children's engagement and played a role in the positive results.

    “Indeed, children were only asked to do this task during one short session which may have facilitated their engagement and played a role in the positive results we saw.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given
    § 5.3.1 Percentage of divergent questions generated · p. 19

    All data was anonymized so the coder could only see randomly assigned identifiers.

    “All data was anonymized: the coder could only see the identifiers that children were given randomly at the beginning of the intervention.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductown data
    § 4.1 Participants · p. 14

    The groups were assigned with a pseudo-randomized method after collecting profile data.

    “The groups were assigned with a pseudo-randomized method after collecting profile data regarding the age, the device use frequency, the curiosity-related measures, the reading score and their domain-knowledge scores”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data
    § 5.3.2 Syntactic scores for the questions generated · p. 19

    The inter-rater reliability for syntactic scoring of questions was 76.8%.

    “they had an inter-rater reliability of 76.8%.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductnone given
    § 7.4.1 On the divergent QA fluency · p. 27

    Hypotheses concerning the absence of outliers and normality of the divergent QA-fluency scores were verified before running the test.

    “Hypotheses concerning the absence of outliers and normality of the divergent QA-fluency scores were verified pre-running the test.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given
    § 9 Limitations and future directions · p. 29

    It is unknown whether enhanced perceptions of self-efficacy are directly connected to children's perception of their performance or to other factors.

    “Indeed, and even though we see enhanced perceptions of the self-efficiency, we do not know if this is directly connected to their perception of their performance or to other factors, like the repeatability of the exercise, bias in the self-report measure, etc.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Study design and conductown data
    § 8 Discussion · p. 28

    The approach was validated by using human annotations to score the cues and by comparing their impact on children's divergent QA performances with human-based interventions.

    “We validate our approach by using human annotations to score these cues and by comparing their impact on children's divergent QA performances with human-based interventions.”
    Quote in the paperSubstantial claim

  • May combine two claims. The quote only partly states this claim.

    Study design and conductown data
    § 4.1 Participants · p. 14

    The three groups were balanced and did not differ on initial profile measures (age, device use frequency, curiosity trait, perception of curiosity, reading ability, ability to ask divergent questions, domain-knowledge quiz score), except for familiarity with using digital tools (device use frequency p=0.008).

    “As shown in Table 1, we had three balanced groups that were not different in terms of these initial profile measures, with the exception of the familiarity with using digital tools measure.”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: The difference between groups in familiarity with using digital tools does not have a significant effect on the measure of interest (formulating divergent questions).

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationown data
    § 3.3.1 Generating the agents' cues using chain-of-thoughts prompting in GPT-3 · p. 12

    GPT-3 at the scale used is sufficiently robust on the prompt format dimension.

    “probably because LLM at the scale of GPT-3 is sufficiently robust on this format dimension.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 5.1 Children profile measures · p. 18

    The divergent QA fluency test used was the same as in previous studies [1, 2].

    “This same divergent QA fluency test was used in previous studies in [1, 2].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [1] Abdelghani, 2022, Conversational agents for training curiosity-driven learning in children, International Journal of Human-Computer Studies source

      Not checked in the free run.

    • cites [2] Alaimi, M., Law, E., Pantasdo, K. D., Oudeyer, P. Y., & Sauzéon, H. (2020). Pedagogical agents for fostering question-asking skills in children. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (pp. 1–13). New York, NY: Association for Computing Machinery. source

      Not checked in the free run.

  • May combine two claims.

    Interpretationown data
    § 7.1.3 Generating the semantic cues · p. 22

    The prompting approaches were successful in generating divergent semantic cues that are closely related to the context and comparable to the hand-crafted condition.

    “Similarly to the linguistic cues, here we also see results suggesting that our prompting approaches were successful in generating divergent semantic cues that are closely related to the context in question and are comparable to what we had in the hand-crafted condition.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductcitation
    § D Syntactic scores for evaluating children's questions · p. 32

    Syntactic quality of children's questions was assessed using a grid inspired by Aschner's classification.

    “In assessing the questions syntactic quality, we use the grid inspired from Aschner's classification in [14].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [14] Gallagher, 1963, A preliminary report on analyses of classroom interaction, Merrill-Palmer Quarterly

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Premisecitation
    § 3.2.2 Choice of the cues proposed by the agent · p. 11

    Lack of control over the task might alter children's sense of autonomy and competency, which are central factors for motivation and curiosity according to self-determination theory.

    “this lack of control over the task might also alter with children's sense of autonomy and competency, which are central factors for motivation and curiosity according to the Self-Determination Theory( [48], [17]).”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [48] Raffel, 2020, Exploring the limits of transfer learning with a unified text-to-text transformer, Journal of Machine Learning Research

      Not checked in the free run.

    • cites [17] Goupil, 2023, Curiosity as a metacognitive feeling, Cognition source

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Interpretationown data
    § 7.4.2 On the perception of curiosity · p. 27

    Giving children tasks with a larger margin of choice and autonomy can affect their perception of self-efficacy.

    “This result aligns well with what's discussed in the design rationale section 3.2.2 where we suggest that giving children tasks with a larger margin of choice and autonomy can effect their perception of their self-efficiency.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data
    § 5.3.1 Percentage of divergent questions generated · p. 19

    The inter-rater reliability for coding the percentage of divergent questions was 87.6%.

    “The inter-rater reliability was of 87.6%.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductcitation
    § 5.4.2 Perception of curiosity · p. 19

    The perception of curiosity was assessed using Post's validated CIAC questionnaire [43].

    “we use Post's [43] validated CIAC questionnaire.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [43] Metcalfe, 2020, Epistemic curiosity and the region of proximal learning, Current Opinion in Behavioral Sciences source

      Not checked in the free run.

46 of 46

  • The paper relies on this without support.

    Generating the curiosity-prompting cues by hand for each educational resource is a very long and costly process, which makes the generalization and scale-up of the approach difficult.

    “this method is still limited since it relies on generating the said prompts by hand for each educational resource, which can be a very long and costly process.”
    “However, and as mentioned above, this manual implementation, being very time-costly, makes the generalization and scale-up of the approach, even though pedagogically efficient, very difficult.”

  • The paper relies on this without support.

    The previous hand-crafted conversational agents had very low intelligence and computational power, which limited their usefulness, usability and transfer to different classroom activities.

    “However, the agents presented in these tools had very low intelligence level (i.e. guided by pre-defined automatons) and restrained children to think about predefined specific questions which can limit their usefulness, usability and transfer in different classroom activities.”
    “these previous implementations presented important drawbacks, mainly concerning the difficulty of their use on a larger scale and in different contexts, given that the agents had low computational powers.”

  • The authors state this as an assumption.

    The difference between groups in familiarity with using digital tools does not have a significant effect on the measure of interest (formulating divergent questions).

    “we assume that this difference does not have a significant effect of our measure of interest, i.e. formulating divergent questions.”

  • The paper relies on this without support.

    Practitioners in real-world pedagogical applications are less likely to have machine learning expertise or sufficient computational resources to fine-tune an LLM or optimize outputs with example combinations.

    “practitioners are less likely to have machine learning expertise nor sufficient computational resources to fine-tune an LLM, or to optimize the LLM outputs by providing the model with different combinations of example.”

  • The paper relies on this without support.

    The study has no control over GPT-3's outputs.

    “As we have no control over GPT-3's outputs, we choose not to put children in direct contact with the model in order to avoid any potential offensive interactions.”

  • The paper relies on this without support.

    The open agent's cues (keywords) do not rely on the words themselves for their divergence level but on how children choose to use them.

    “However, this is not the case for the open agent's cues, i.e., the keywords, as the questions' divergence levels here do not rely on the words themselves but on how children choose to use them.”

  • The paper relies on this without support.

    Children's ability to ask divergent questions is an important aspect of their epistemic curiosity and a relevant behavioral indicator for it.

    “Children's ability to ask divergent questions is an important aspect of their epistemic curiosity and can be seen as a relevant behavioral indicator for it.”

  • The paper relies on this without support.

    The open agent leaves more space for children to formulate questions of their own choice and thus to express their own curiosity.

    “given that it leaves more space to children to formulate questions of their own choice and thus, to express their own curiosity”
8 of 8

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • GenAI broadens forms of expression, strengthens creativity, and supports higher-order and critical thinking.

    “which broadens forms of expression, strengthens creativity [4,5], and supports higher-order and critical thinking [6,7].”

    From A Systematic Review of Generative AI in K–12: Mapping Goals, Activities, Roles, and Outcomes via the 3P Model (2025) · cites this paper as [6]

    The citation [6,7] is attached to the claim about higher-order and critical thinking, not the preceding claim about creativity. The paper describes divergent questions as high-level questions requiring hypotheses and inferences (Section 2.1, p. 4), and reports improved divergent question-asking after training (Section 7.4.1, p. 26). It does not establish an improvement in critical thinking, so that part is unsupported. (LLM, conf=0.85)

    Experimental — accuracy not yet measured

  • AI often acts as a dialogic coach or teachable agent that organizes a sequence of demonstration, explanation, questioning, and feedback.

    “Beyond answering, AI often acts as a dialogic coach or teachable agent that organizes a sequence of demonstration, explanation, questioning, and feedback, as illustrated by the APLUS mathematics environment and curiosity question training agents [6,102,128].”

    From A Systematic Review of Generative AI in K–12: Mapping Goals, Activities, Roles, and Outcomes via the 3P Model (2025) · cites this paper as [6]

    The Study design (p. 7) and Procedure (p. 15) describe conversational agents that guide children in forming questions through dialogue and linguistic and semantic cues. They support the curiosity-question-training example, but do not describe an agent-led sequence of demonstration, explanation, questioning, and feedback. (LLM, conf=0.84)

    Experimental — accuracy not yet measured

  • Numerous studies report higher enthusiasm, participation, curiosity, and satisfaction after AI use, alongside a general preference for AI tools as efficient and convenient.

    “Numerous studies report higher enthusiasm, participation, curiosity, and satisfaction after AI use [6,70,74,81,87,91,92,101,104,106,113,114,116,119,131,141,142], alongside a general preference for AI tools as efficient and convenient [99,111,124,127,135,140].”

    From A Systematic Review of Generative AI in K–12: Mapping Goals, Activities, Roles, and Outcomes via the 3P Model (2025) · cites this paper as [6]

    The Results (pp. 26–27) report improved divergent question-asking and a change in children’s perception of curiosity after the intervention. The Discussion (p. 29) reports high but similar motivation across conditions; this paper does not establish increased enthusiasm, participation, or satisfaction. The efficient-and-convenient preference belongs to a separate citation group, not this source. (LLM, conf=0.86)

    Experimental — accuracy not yet measured

  • LLM-based agents need explicit design to cultivate questioning behaviors.

    “This scarcity-value effect aligns with Abdelghani et al.'s [1] observation that LLM-based agents need explicit design to cultivate questioning behaviors.”

    From OpenClaw AI Agents as Informal Learners at Moltbook: Characterizing an Emergent Learning Community at Scale (2026) · cites this paper as [1]

Statistics recomputed

Every reported test result, recomputed from its test statistic and degrees of freedom. A result shows as not adding up only once an independent check has confirmed it.

  • ConsistentF(2, 72) = 2.23reported p = .11 · recomputed p = .115

    “F(2,72)=2.23, p = .11”

1 of 1

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Pauline Lucas appears in only one author list

    Minor

    The indexed title page lists Pauline Lucas as an author, but the paper's subsequent author line omits her.

1 of 1

Cites

Flagged first, then by citations

Research works printed but not linked: 10

  1. Aleven, V., Koedinger, K. R., & Cross, K. (1999). Tutoring answer explanation fosters learning with understanding. In Proceedings of the 9th International Conference on Artificial Intelligence in Education (pp. 199–206).
  2. Berlyne, 1954, A theory of human curiosity, British Journal of Psychology, 45, 180
  3. Devlin, J., Chang, M. W., Lee, K., & Toutanova, K. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805. · arXiv
  4. Gallagher, 1963, A preliminary report on analyses of classroom interaction, Merrill-Palmer Quarterly, 9, 183
  5. Post, 2015, Development and validation of a questionnaire to measure primary school children’s images of and attitudes towards curiosity (the CIAC questionnaire), Motivation and Emotion, 42, 159, 10.1007/s11031-018-9728-9 · DOI
  6. Reio, 2000, Field investigation of the relationship among adult curiosity, workplace learning, and job performance, Human Resource Development Quarterly, 11, 5, 10.1002/1532-1096(200021)11:1<5::AID-HRDQ2>3.0.CO;2-A · DOI
  7. Silvervarg Flycht-Eriksson, A. S., and Jönsson, A. (2010). Towards a conversational pedagogical agent capable of affecting attitudes and self-efficacy. In Proceedings of the Second Workshop on Natural Language Processing in Support of Learning: Metrics, Feedback and Connectivity
  8. Stumm, 2011, The hungry mind – intellectual curiosity is the third pillar of academic performance, Perspectives on Psychological Science, 6, 574, 10.1177/1745691611421204 · DOI
  9. Vallerand, 1989, Construction et validation de l’échelle de motivation en éducation (EME), Canadian Journal of Behavioural Science / Revue Canadienne Des Sciences Du Comportement, 21, 323
  10. Zhao, Z., Wallace, E., Feng, S., Klein, D., and Singh, S. (2021) Calibrate before use: Improving few-shot performance of language models. In International Conference on Machine Learning, pages 12697–12706. PMLR

Cited by

Sorted by citations

Authors
Abdelghani, R., Wang, Y.-H., Yuan, X., Wang, T., Lucas, P., Sauzéon, H., & Oudeyer, P.-Y.
Affiliations
  • Institut national de recherche en sciences et technologies du numérique (FR)
  • National Chung Hsing University (TW)
  • Microsoft Research Montréal (Canada) (CA)
  • Université de Bordeaux (FR)
Corresponding author
Abdelghani, Rania
ORCID
Venue
International Journal of Artificial Intelligence in Education
Publisher
Elsevier BV
Published
2024-06
Volume
34
Issue
2
Pages
483-518
ISSN
1560-4292 (ISSN-L)
DOI
10.1007/s40593-023-00340-7
OpenAlex
W4382656966
Cited by
129
References
52
Citations by year
2026: 24 · 2025: 55 · 2024: 37 · 2023: 13
FWCI
12.13
Type
journal article
Language
en
Open access
Licence
Locations
  • International Journal of Artificial Intelligence in Education (publishedVersion, cc-by-nc-nd)
  • HAL (submittedVersion, cc-by-nc)
  • HAL (submittedVersion, other-oa)
  • Oskar-Bordeaux (Universite de Bordeaux) (submittedVersion)
Funders
  • Association Nationale de la Recherche et de la Technologie
  • EvidenceB
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Intelligent Tutoring Systems and Adaptive Learning
  • Physical Sciences › Computer Science › Information Systems › Artificial Intelligence in Education
  • Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychological and Educational Research Studies
Keywords
divergent questions, educational agents, GPT-3, large language models, elementary school students

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-06.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-06.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review