When the Robotic Maths Tutor is Wrong - Can Children Identify Mistakes Generated by ChatGPT?
Open accessAbstract
This study delves into integrating Large Language Models (LLMs), particularly ChatGPT-powered robots, as educational tools in primary school mathematics. Against the backdrop of Artificial Intelligence (AI) increasingly permeating educational settings, our investigation focuses on the response of young learners to errors made by these LLM-powered robots. Employing a user study approach, we conducted an experiment using the Pepper robot in a primary school classroom environment, where 77 primary school students from multiple grades (Year 3 to 5) took part in interacting with the robot. Our statistically significant findings highlight that most students, regardless of the year group, could discern between correct and incorrect responses generated by the robots, demonstrating a promising level of understanding and engagement with the AI-driven educational tool. Additionally, we observed that students' correctness in answering the Maths questions significantly influenced their ability to identify errors, underscoring the importance of prior knowledge in verifying LLM responses and detecting errors. Additionally, we examined potential confounding factors such as age and gender. Our findings underscore the importance of gradually integrating AI-powered educational tools under the guidance of domain experts following thorough verification processes. Moreover, our study calls for further research to establish best practices for implementing AI-driven pedagogical approaches in educational settings,
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Enhancing students' critical thinking and real-world problem-solving practices remains a key objective, with GenAI-supported interactive learning and analytical tasks designed to promote critical analysis, logical reasoning, and the generative use of knowledge to authentic challenges in science and mathematics.
“Additionally, enhancing students' critical thinking and real-world problem-solving practices remains a key objective. GenAI-supported interactive learning and analytical tasks are designed to promote critical analysis, logical reasoning, and the generative use of knowledge to authentic challenges in science and mathematics.”
From A Systematic Review of Generative AI in K–12: Mapping Goals, Activities, Roles, and Outcomes via the 3P Model (2025) · cites this paper as [105]
Split into 2 atoms — 1 confirmed, 1 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.
Experimental — accuracy not yet measuredStudies document limited understanding of model training and datasets, which impairs students' ability to detect errors or deceptive content and leads to weak knowledge/analysis or no significant improvement.
“Studies document limited understanding of model training and datasets [112,122,135,136], which impairs students' ability to detect errors or deceptive content [105,133] and leads to weak knowledge/analysis [109] or no significant improvement [98].”
From A Systematic Review of Generative AI in K–12: Mapping Goals, Activities, Roles, and Outcomes via the 3P Model (2025) · cites this paper as [105]
The Robot Error section reports that 72% of students identified incorrect answers and that partially correct answers caused some confusion, supporting the point that detecting AI errors can be difficult. The paper does not establish that limited understanding of model training or datasets caused that difficulty, and it does not test detection of deceptive content. The later claims about weak knowledge or no significant improvement are attached to other citations. (LLM, conf=0.85)
Experimental — accuracy not yet measuredIn school settings, students repeatedly interact with AI assistants across English writing, reading comprehension, mathematics problem-solving, physics, quantum, and cognitive science, and computer programming.
“In school settings, students repeatedly interact with AI assistants (such as ChatGPT, custom bots, and social robots) across English writing [85], reading comprehension [76,79,80,107,109], mathematics problem-solving [99,105], physics, quantum, and cognitive science [78,91,104,127], and computer programming [97,98,140,141].”
From A Systematic Review of Generative AI in K–12: Mapping Goals, Activities, Roles, and Outcomes via the 3P Model (2025) · cites this paper as [105]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · cited 1× · see citing works
“…Pathways Language Model (PaLM), Facebook Open Pre-trained Transformers (OPT), Microsoft BARD and OpenAI Generative Pre-trained Transformers (GPT) have managed to process and generate large amounts of text after training on the very large corpus from the internet-wide content and other sources [30].”
BackgroundWhat Is the Impact of ChatGPT on Education? A Rapid Review of the Literature
Lo, C. K. (2023) · Education Sciences · doi:10.3390/educsci13040410 · Cited by 2,024 · cited 1× · see citing works
“ChatGPT, as a famous LLM example, showed out-standing performance enhancement in supporting students in economics, satisfactory in programming, and unsatisfactory in mathematics [12].”
BackgroundGender differences in mathematics performance: A meta-analysis.
Hyde, J. S., Fennema, E., & Lamon, S. J. (1990) · Psychological Bulletin · doi:10.1037/0033-2909.107.2.139 · Cited by 1,732 · cited 1× · see citing works
“The findings of the fourth research question align with other studies such as [35].”
ResultSelf-Esteem Development Across the Lifespan
Robins, R. W., & Trzesniewski, K. H. (2005) · Current Directions in Psychological Science · doi:10.1111/j.0963-7214.2005.00353.x · Cited by 748 · cited 1× · see citing works
“This confidence in children is known to decrease as people get older [34].”
BackgroundAlqahtani, T., Badreldin, H. A., Alrashed, M., Alshaya, A. I., Alghamdi, S. S., bin Saleh, K., Alowais, S. A., Alshaya, O. A., Rahman, I., Al Yami, M. S., et al. (2023) · Research in Social and Administrative Pharmacy · doi:10.1016/j.sapharm.2023.05.016 · Cited by 565 · see citing works
Unlocking the opportunities through ChatGPT Tool towards ameliorating the education system
Javaid, M., Haleem, A., Singh, R. P., Khan, S., & Khan, I. H. (2023) · BenchCouncil Transactions on Benchmarks, Standards and Evaluations · doi:10.1016/j.tbench.2023.100115 · Cited by 535 · see citing works
Artificial Intelligence (AI) Literacy in Early Childhood Education: The Challenges and Opportunities
Su, J., Ng, D. T. K., & Chu, S. K. W. (2023) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2023.100124 · Cited by 489 · cited 1× · see citing works
“Other studies discussed further challenges and opportunities in AI in early childhood education [18].”
BackgroundYang, W. (2022) · Computers and Education Artificial Intelligence · doi:10.1016/j.caeai.2022.100061 · Cited by 440 · cited 1× · see citing works
“The “AI for Kids” proposal emphasizes the importance of exposing children to AI literacy as part of digital literacy for intelligent societies [17].”
BackgroundElkhatat, A. M., Elsaid, K., & Almeer, S. (2023) · International Journal for Educational Integrity · doi:10.1007/s40979-023-00140-5 · Cited by 392 · cited 1× · see citing works
“Some other studies evaluated meth-ods to detect AI generated vs Human content such as Writer, Copyleaks, GPTZero, and CrossPlag [8].”
BackgroundStade, E. C., Stirman, S. W., Ungar, L. H., Boland, C. L., Schwartz, H. A., Yaden, D. B., Sedoc, J., DeRubeis, R. J., Willer, R., & Eichstaedt, J. C. (2024) · npj Mental Health Research · doi:10.1038/s44184-024-00056-z · Cited by 391 · cited 2× · see citing works
“The rush to AI-answers in various tasks at all ages and disciplines has created concerns over creativity loss and negative impacts on learning and natural intelligence development and mastery of skills [14]– [16].”
BackgroundThe ethics of disclosing the use of artificial intelligence tools in writing scholarly manuscripts
Hosseini, M., Resnik, D. B., & Holmes, K. (2023) · Research Ethics · doi:10.1177/17470161231180449 · Cited by 253 · cited 1× · see citing works
“The downside of using AI for gaining such merits could be that the AI skills are being evaluated and not the long-term student’s skills development [10].”
BackgroundENAI Recommendations on the ethical use of Artificial Intelligence in Education
Foltynek, T., Bjelobaba, S., Glendinning, I., Khan, Z. R., Santos, R., Pavletic, P., & Kravjar, J. (2023) · International Journal for Educational Integrity · doi:10.1007/s40979-023-00133-4 · Cited by 180 · cited 1× · see citing works
“Some studies raised questions of integrity when AI authors students’ submissions or academic manuscripts for scientific publications or academic and educational gains [9].”
BackgroundBernabei, M., Colabianchi, S., Falegnami, A., & Costantino, F. (2023) · Computers and Education Artificial Intelligence · doi:10.1016/j.caeai.2023.100172 · Cited by 175 · cited 1× · see citing works
“Some publications report that the submissions are enhanced but do not focus on student skills [11].”
BackgroundChatGPT has entered the classroom: how LLMs could transform education
Extance, A. (2023) · Nature · doi:10.1038/d41586-023-03507-3 · Cited by 127 · cited 2× · see citing works
“The work in [5] emphasises the importance of verifying the LLM erroneous responses that are labelled as hallucinations.”
BackgroundThe Effectiveness of Educational Robots in Improving Learning Outcomes: A Meta-Analysis
Wang, K., Sang, G., Huang, L.-Z., Li, S., & Guo, J.-W. (2023) · Sustainability · doi:10.3390/su15054637 · Cited by 102 · cited 1× · see citing works
“Robots also have been frequently used in educational contexts [21], e.g. to support children with autism in their learning [22].”
BackgroundArtificial intelligence-based robots in education: A systematic review of selected SSCI publications
Chu, S.-T., Hwang, G.-J., & Tu, Y.-F. (2022) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2022.100091 · Cited by 88 · cited 1× · see citing works
“AI-powered robots’ role in education and the learning impact has been evaluated in a systematic review [23].”
BackgroundMalanchini, M., Rimfeld, K., Allegrini, A. G., Ritchie, S. J., & Plomin, R. J. (2020) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2020.01.016 · Cited by 80 · see citing works
Rolison, J. J., Morsanyi, K., & Peters, E. M. (2020) · Medical Decision Making · doi:10.1177/0272989x20904725 · Cited by 64 · cited 1× · see citing works
“Building confidence in mathematical skills and identifying errors becomes essential to address these challenges [31], [32].”
BackgroundCan Students without Prior Knowledge Use ChatGPT to Answer Test Questions? An Empirical Study
Shoufan, A. (2023) · ACM Transactions on Computing Education · doi:10.1145/3628162 · Cited by 59 · cited 2× · see citing works
“While other work has illustrated various shortcomings in LLM-assisted learning of computer science without prior knowledge [13].”
BackgroundA review of assessment for learning with artificial intelligence
Memarian, B., & Doleck, T. (2023) · Computers in Human Behavior Artificial Humans · doi:10.1016/j.chbah.2023.100040 · Cited by 52 · cited 2× · see citing works
“The assessment for learning (AFL) link with AI-powered education has been systematically reviewed, exposing some challenges that still need to be addressed in the literature [16].”
BackgroundZhao, F., & Miao, D. (2023) · Communications in computer and information science · doi:10.1007/978-981-99-7587-7 · Cited by 49 · see citing works
Kaspar, the social robot and ways it may help children with autism – an overview
Robins, B., Dautenhahn, K., & Nadel, J. (2018) · Enfance · doi:10.3917/enf2.181.0091 · Cited by 31 · cited 1× · see citing works
“Robots also have been frequently used in educational contexts [21], e.g. to support children with autism in their learning [22].”
BackgroundSeaba, V. (2023) · OALib · doi:10.4236/oalib.1110763 · Cited by 23 · cited 2× · see citing works
“While fine-tuning these models into specific curriculum content can enhance the responses personalised to different educational contexts [29], choice of training dataset and other biases can still negatively impact the validity of the responses and might introduce other mistakes [7].”
BackgroundIsmail, H., Hussein, N., Harous, S., & Khalil, A. (2023) · Education Sciences · doi:10.3390/educsci13070741 · Cited by 22 · cited 1× · see citing works
“AI-powered applications began offering personalised learning experiences [26], intelligent tutoring systems [27], and automated grading [28].”
BackgroundMachine Learning-Based Automated Grading and Feedback Tools for Programming: A Meta-Analysis
Messer, M., Brown, N. C. C., Kölling, M., & Shi, M. (2023) · ACM Conference on Innovation and Technology in Computer Science Education (ITiCSE) · doi:10.1145/3587102.3588822 · Cited by 20 · cited 1× · see citing works
“AI-powered applications began offering personalised learning experiences [26], intelligent tutoring systems [27], and automated grading [28].”
BackgroundSchool students’ confidence when answering diagnostic questions online
Foster, C. F., Woodhead, S., Barton, C., & Clark‐Wilson, A. (2021) · Educational Studies in Mathematics · doi:10.1007/s10649-021-10084-7 · Cited by 17 · cited 1× · see citing works
“Building confidence in mathematical skills and identifying errors becomes essential to address these challenges [31], [32].”
BackgroundEmpowering Teachers to Build a Better World
Reimers, F. (2020) · Springer briefs in education · doi:10.1007/978-981-15-2137-9 · Cited by 17 · see citing works
Supporting Mathematics Teaching for Mastery in England
Blausten, H., Gyngell, C., Aichmayr, H., & Spengler, N. (2020) · Springer briefs in education · doi:10.1007/978-981-15-2137-9_2 · Cited by 12 · cited 1× · see citing works
“It is a common interest of the UK government to evaluate all factors affecting Maths education, and study the various aspects of enhancing the UK international ratings [36].”
BackgroundAn Antropomorphic Robot with ChatGPT for Learning Activities: The Teachers' Perspective
Sacco, F., Rossini, G., Manzi, F., Dio, C. D., Aquilino, L., Cangelosi, A., Raggioli, L., Massaro, D., & Marchetti, A. (2023) · IEEE International Conference on Metrology for eXtended Reality, Artificial Intelligence and Neural Engineering (MetroXRAINE) · doi:10.1109/metroxraine58569.2023.10405596 · Cited by 8 · cited 1× · see citing works
“Another primary school experiment is teachers-focused [25].”
Chat Failures and Troubles: Reasons and SolutionsPreprint
Helal, M., Holthaus, P., Lakatos, G., & Amirabdollahian, F. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2309.03708 · Cited by 2 · see citing works
Self-citation
Flagged first, then by citations
Research works printed but not linked: 6
- C. Wang, X. Liu, Y. Yue, X. Tang, T. Zhang, C. Jiayang, Y. Yao, W. Gao, X. Hu, Z. Qi, Y. Wang, L. Yang, J. Wang, X. Xie, Z. Zhang, and Y. Zhang, "Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificity," Dec. 2023, arXiv:2310.07521 [cs]. [Online]. Available: http://arxiv.org/abs/2310.07521 · arXiv
- J. Ye, X. Chen, N. Xu, C. Zu, Z. Shao, S. Liu, Y. Cui, Z. Zhou, C. Gong, Y. Shen, J. Zhou, S. Chen, T. Gui, Q. Zhang, and X. Huang, "A Comprehensive Capability Analysis of GPT-3 and GPT-3.5 Series Models," Dec. 2023, arXiv:2303.10420 [cs]. [Online]. Available: http://arxiv.org/abs/2303.10420 · arXiv
- F. Zeng, W. Gan, Y. Wang, N. Liu, and P. S. Yu, "Large Language Models for Robotics: A Survey," Nov. 2023, arXiv:2311.07226 [cs]. [Online]. Available: http://arxiv.org/abs/2311.07226 · arXiv
- A. Alkhatlan and J. Kalita, "Intelligent Tutoring Systems: A Comprehensive Historical Survey with Recent Develop- ments," Dec. 2018, arXiv:1812.09628 [cs]. [Online]. Available: http://arxiv.org/abs/1812.09628 · arXiv
- X. Liu, J. Wang, J. Sun, X. Yuan, G. Dong, P. Di, W. Wang, and D. Wang, "Prompting Frameworks for Large Language Models: A Survey," Nov. 2023, arXiv:2311.12785 [cs]. [Online]. Available: http://arxiv.org/abs/2311.12785 · arXiv
- Y. Zhang, Y. Luo, Y. Yuan, and A. C.-C. Yao, "Autonomous Data Selection with Language Models for Mathematical Texts," Apr. 2024, arXiv:2402.07625 [cs]. [Online]. Available: http://arxiv.org/abs/2402.07625 · arXiv
Cited by
Sorted by citations
Latif, E., Liu, V., & Zhaı, X. (2026) · Smart Learning Environments · doi:10.1186/s40561-025-00427-9 · Cited by 78 · see citing works
“An example of a system that exemplifies the verbal tutoring system category is Helal et al. (2024), where the robot speaks and displays questions for the student to answer.”
Lin, X., & Tan, H. (2025) · Systems · doi:10.3390/systems13100840 · Cited by 22 · see citing works
Mathematic Problems Categorization Using Synthetic- LLM Models
Rahravi, M., Khouri, M., & Ghasemzadeh, M. (2024) · International Conference on Signal Processing and Intelligent Systems (ICSPIS) · doi:10.1109/icspis65223.2024.10931108
“Hilal et al. [9] explore the integration of LLMs, specifically ChatGPT-powered robots, as educational tools in primary school mathematics.”
LLM-Powered Social Robots for an Indoor Navigation Assistant
Carnaz, G., FARIA, E. G., & Menezes, P. J. (2025) · Experiment@International Conference · doi:10.1109/exp.at2565440.2025.11348564
“…applications that involve the use of LLMs in different contexts, such as facilitating the exchange of health information between parents and young adults [2], supporting cognitive stimulation therapy [7], or enhancing primary school mathematics education through ChatGPT-powered social robots [8].”
Orlock: A Modular Agent for Natural Interaction in Indoor Navigation
Carnaz, G., FARIA, E. G., & Menezes, P. J. (2025) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/smc58881.2025.11343491
“Prior works have explored their usein primary education through ChatGPT-enhanced robotics [5], and in hospitals, where LLM-powered robots support patient interactions in waiting rooms [1].”
Intawong, K., Ariya, P., Khanchai, S., Thongthip, P., & Puritat, K. (2026) · TechTrends · doi:10.1007/s11528-026-01201-5
“These behaviors align with broader findings from Helal et al. (2024) and Poushneh and Vasquez-Parraga (2024), who observed that male users often seek exploratory and entertainment-driven engagement with technology, while female users prioritize task efficiency and clarity.”
- Authors
- Helal, M., Holthaus, P., Wood, L., Velmurugan, V., Lakatos, G., Moros, S., & Amirabdollahian, F.
- Affiliations
- University of Hertfordshire (GB)
- School of Physics, Engineering and Computer Science, University of Hertfordshire,College Lane, Hatfield,UK,AL10 9AB
- ORCID
- Venue
- 2024 5th International Conference on Artificial Intelligence, Robotics and Control (AIRC)
- Publisher
- IEEE
- Published
- 2024-04-22
- Pages
- 83-90
- OpenAlex
- W4402572612
- Cited by
- 6
- References
- 29
- Citations by year
- 2026: 2 · 2025: 3 · 2024: 1
- FWCI
- 2.28
- Type
- conference paper
- Language
- en
- Open access
- green
- hdl.handle.net
- Locations
- International Conference on Artificial Intelligence, Robotics and Control (AIRC) (publishedVersion)
- University of Hertfordshire Research Archive (University of Hertfordshire) (acceptedVersion)
- University of Hertfordshire Research Archive (University of Hertfordshire) (submittedVersion, cc-by)
- Topics
- Physical Sciences › Computer Science › Information Systems › Artificial Intelligence in Education
- Physical Sciences › Computer Science › Artificial Intelligence › Intelligent Tutoring Systems and Adaptive Learning
- Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
- Keywords
- ChatGPT, large language models, primary school mathematics, error detection, Pepper robot, prior knowledge, age and gender differences, artificial intelligence in education
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
