CoChat
Sign Up
Cited by 14Artificial Intelligence in EducationIntelligent Tutoring Systems and Adaptive LearningTopic Modeling

The DOI matches its Crossref record; 24 of the 24 references it cites were found in a registry or an index.

LLMs Performance in Answering Educational Questions in Brazilian Portuguese: A Preliminary Analysis on LLMs Potential to Support Diverse Educational Needs

Open access
Rodrigues, Luiz, Xavier, Cleon, Costa, Newarney, Batista, Hyan, Silva, Luiz Felipe Bagnhuk, Chaleghi de Melo, Weslei, Gasevic, Dragan, Ferreira Mello, Rafael·2025·Proceedings of the 15th International Learning Analytics and Knowledge Conference
DOI 10.1145/3706468.3706515
Open this paper in CoChat — free account

Abstract

Question-answering systems facilitate adaptive learning and respond to student queries, making education more responsive. Despite that, challenges such as natural language understanding and context management complicate their widespread adoption, where Large Language Models (LLMs) offer a promising solution. However, existing research is predominantly focused on English, proprietary models, and often limited to a single question type, subject, or skill, leaving a gap in understanding LLMs' performance in languages like Brazilian Portuguese and across questions of various characteristics. This study investigates how LLMs could be integrated in an educational question-answering system efficiently to answer different question types (multiple-choice, cloze, open-ended), subjects (mathematics and Portuguese language), and skills (summation/subtraction, multiplication, interpretation, and grammar), evaluating answers by GPT-4 - the main LLM at the time of writing - and Sabiá - the open-source Brazilian Portuguese LLM - based on grades assigned by two experienced teachers. Overall, both LLMs demonstrated strong overall performance, with mean scores close to 9.8 out of 10. However, specific challenges emerged, with distinct strengths and weaknesses observed for each model, such as GPT-4's error in a multiple-choice subtraction question and Sabiá's misinterpretation of a cloze question.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3706468.3706515

Cites

24 of 24 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • LLMs have demonstrated substantial capabilities in educational practice, e.g., automatic grading, answer generation, and student-chatbot interaction.

    “Large Language Models (LLMs) have demonstrated substantial capabilities in educational practice, e.g., automatic grading (Ferreira Mello et al., 2025), answer generation (Rodrigues et al., 2025), and student-chatbot interaction (Woollaston et al., 2025; Hao et al., 2026; Zhang et al., 2025).”

    From Assessing the Impact and Underlying Pathways of Sequenced AI feedback on Student Learning (2026) · cites this paper as Rodrigues et al., 2025

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Flagged first, then by citations

Research works printed but not linked: 4

  1. Xiaoyu Bai and Manfred Stede. 2022. A Survey of Current Machine Learning Approaches to Student Free-Text Evaluation for Intelligent Tutoring. International Journal of Artificial Intelligence in Education (2022) 1–39.
  2. Shervin Minaee Tomas Mikolov Narjes Nikzad Meysam Chenaghlu Richard Socher Xavier Amatriain and Jianfeng Gao. 2024. Large language models: A survey. arXiv preprint arXiv:https://arXiv.org/abs/2402.06196 (2024). · arXiv
  3. Jules White Quchen Fu Sam Hays Michael Sandborn Carlos Olea Henry Gilbert Ashraf Elnashar Jesse Spencer-Smith and Douglas C Schmidt. 2023. A prompt pattern catalog to enhance prompt engineering with chatgpt. arXiv preprint arXiv:https://arXiv.org/abs/2302.11382 (2023). · arXiv
  4. Wayne Xin Zhao Kun Zhou Junyi Li Tianyi Tang Xiaolei Wang Yupeng Hou Yingqian Min Beichen Zhang Junjie Zhang Zican Dong et al. 2023. A survey of large language models. arXiv preprint arXiv:https://arXiv.org/abs/2303.18223 (2023). · arXiv

Cited by

Sorted by citations

Related

Metadata

Authors
Rodrigues, L., Xavier, C., Costa, N., Batista, H., Silva, L. F. B., Chaleghi de Melo, W., Gasevic, D., & Ferreira Mello, R.
Affiliations
  • Universidade Federal de Alagoas (BR)
  • Instituto Federal Goiano (BR)
  • Instituto Federal de Educação, Ciência e Tecnologia Goiano - IF Goiano, Iporá, Brazil
  • Universidade Federal de Pernambuco (BR)
  • Faculdade de Campina Grande do Sul, Campina Grande do Sul, Brazil
  • Universidade Tecnológica Federal do Paraná, Curitiba, Brazil
  • Monash University, Melbourne, Australia
ORCID
Venue
Proceedings of the 15th International Learning Analytics and Knowledge Conference
Publisher
ACM
Published
2025-03-03
Pages
865-871
DOI
10.1145/3706468.3706515
OpenAlex
W4407842050
Cited by
13
References
24
Citations by year
2026: 6 · 2025: 7
FWCI
15.60
Type
conference paper
Language
en
Open access
Licence
Locations
  • International Conference on Learning Analytics and Knowledge (LAK) (publishedVersion, cc-by)
  • Monash University Research Portal (Monash University) (submittedVersion, cc-by)
  • HAL (submittedVersion)
Funders
  • Conselho Nacional de Desenvolvimento Científico e Tecnológico (310888/2021- 2)
  • OpenAI research Program (1)
Topics
  • Social Sciences › Education › Artificial Intelligence in Education
  • Physical Sciences › Computer Science › Artificial Intelligence › Intelligent Tutoring Systems and Adaptive Learning
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
Keywords
large language models, educational question answering, Brazilian Portuguese, GPT-4, Sabia, multiple-choice questions, cloze questions, open-ended questions, mathematics education, Portuguese language teaching, open-source language models, adaptive learning

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review