Cited by 111Artificial Intelligence in EducationIntelligent Tutoring Systems and Adaptive LearningAI in Service Interactions

Perceived satisfaction of university students with the use of chatbots as a tool for self-regulated learning

Open access
Sáiz-Manzanares, María Consuelo, Marticorena-Sánchez, Raúl, Martín-Antón, Luis Jorge, González Díez, Irene, Almeida, Leandro·2023·Heliyon

Abstract

Chatbots are a promising resource for giving students feedback and helping them deploy metacognitive strategies in their learning processes. In this study we worked with a sample of 57 university students, 42 undergraduate and 15 Master's degree students in Health Sciences. A mixed research methodology was applied. The quantitative study analysed the influence of the variables educational level (undergraduate vs. master's degree) and level of prior knowledge on the frequency of chatbot use (low vs. average), learning outcomes, and satisfaction with the chatbot's usefulness. In addition, we examined whether the frequency of chatbot use depended on students' metacognitive strategies. The qualitative study analysed the students' suggestions for improvement to the chatbot and the type of questions it used. The results indicated that the level of degree being studied influenced the frequency of chatbot use and learning outcomes, with Master's students exhibiting higher levels of both, but levels of prior knowledge only influenced learning outcomes. Significant differences were also found in students' perceived satisfaction with the use of the chatbot, with Master's students scoring higher, but not with respect to the level of prior knowledge. No conclusive results were found regarding frequency of chatbot use and the levels of students' metacognitive strategies. Further studies are needed to guide this research based on the students' suggestions for improvement.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1016/j.heliyon.2023.e12843

Cites

68 of 68 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Master's students demonstrated higher learning gains and used the chatbot more frequently than bachelor's students.

    “The study found that master’s students demonstrated higher learning gains and used the chatbot more frequently than bachelor’s students.”

    From Digital affordances of AI chatbots in nursing education: a systematic review of learning gains and gaps in the evidence (2026) · cites this paper as [28]

    The Results section ('Influence of the type of degree') reports that master’s students scored higher than bachelor’s students on both chatbot-use frequency and learning outcomes. But 'learning gains' implies improvement over time; the Methods ('Instruments') describes learning outcomes as grades on a 0–10 scale, not a measure of gains. Page numbers were not available in our copy. (LLM, conf=0.87)

    Experimental — accuracy not yet measured
  • Personalization (n = 4) by AI chatbots, through timely feedback, has shown a significant impact on students' diagnostic skills, engagement, and frequency of chatbot use.

    “Personalization (n = 4) by AI chatbots, through timely feedback, have shown a significant impact on students’ diagnostic skills, engagement, and frequency of chatbot use (19, 28, 44).”

    From Digital affordances of AI chatbots in nursing education: a systematic review of learning gains and gaps in the evidence (2026) · cites this paper as [28]

    The literature review (page unavailable) says real-time chatbot interaction can help personalize learning and discusses personalized feedback. The Results and Discussion (pages unavailable) report frequency of chatbot use, but do not show that personalization or timely feedback significantly affected that frequency, diagnostic skills, or engagement. The cited paper therefore supports the general personalization premise, not the claimed significant impacts. (LLM, conf=0.86)

    Experimental — accuracy not yet measured
  • Personalization (n = 3) by AI chatbots significantly enhances nursing students' prior knowledge and academic performance.

    “Similarly, personalization (n = 3) by AI chatbots was found to significantly enhance nursing students’ prior knowledge and academic performance (25, 28).”

    From Digital affordances of AI chatbots in nursing education: a systematic review of learning gains and gaps in the evidence (2026) · cites this paper as [28]

    The Results (page unavailable) report significantly better learning outcomes among students with higher prior knowledge, and the Discussion suggests tailoring chatbot responses to prior-knowledge levels. But the study involved Health Sciences students, not specifically nursing students, and did not find that chatbot personalization enhanced either prior knowledge or academic performance. The cited paper also does not establish the claim’s “n = 3.” (LLM, conf=0.82)

    Experimental — accuracy not yet measured
  • Sáiz-Manzanares et al. (28) found that students with moderate prior knowledge and those pursuing master's degrees showed superior learning gains.

    “Sáiz-Manzanares et al. (28), employing a mixed-methods approach, found that students with moderate prior knowledge and those pursuing master’s degrees showed superior learning gains.”
  • Nursing professionals are challenged to quickly adapt to evolving medical knowledge, make accurate decisions in real-case scenarios, and master complex nursing skills.

    “Additionally, with the rapid advancements in nursing research and the advent of new diseases and medications, nursing professionals are challenged to quickly adapt to evolving medical knowledge, make accurate decisions in real-case scenarios, and master complex nursing skills (1, 17, 26–28).”

    From Digital affordances of AI chatbots in nursing education: a systematic review of learning gains and gaps in the evidence (2026) · cites this paper as [28]

    The Introduction discusses changes in university teaching, and the Limitations section identifies the sample as health-sciences students; neither addresses the challenges faced by nursing professionals. Searches for nursing skills, clinical decisions, evolving medical knowledge, new diseases, and medications found no passage supporting the attributed claim. The indexed passages provide section labels but no page numbers. (LLM, conf=0.76)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 3

  1. [25]Reoyo N. Carbonero M.Á. Martín-Antón L.J. Characteristics of teaching effectiveness from the perspectives of teachers and future secondary school teachers Rev. Educ. 376 2017 62 84 10.4438/1988-592X-RE-2017-376-344 · DOI
  2. [32]Cabales V. Muse: scaffolding metacognitive reflection in design-based research Proceedings of CHI '19: CHI Conference on Human Factors in Computing Systems (CHI '19) 2019 (Glasgow, Scotland UK. ACM, New York, NY, USA)
  3. [48]Gupta S. Jagannath K. Aggarwal N. Sridar R. Wilde S. Chen Y. Artificially Intelligent (AI) Tutors in the Classroom: A Need Assessment Study of Designing Chatbots to Support Student Learning vol. 213 2019 PACIS 2019 Proceedings https://aisel.aisnet.org/pacis2019/213

Cited by

Sorted by citations

Authors
Sáiz-Manzanares, M. C., Marticorena-Sánchez, R., Martín-Antón, L. J., González Díez, I., & Almeida, L.
Affiliations
University of Minho (PT)
Venue
Heliyon
Publisher
Elsevier BV
Published
2023-01
Volume
9
Issue
1
Pages
e12843
ISSN
2405-8440 (ISSN-L)
OpenAlex
W4318264071
Cited by
96
References
65
Citations by year
2026: 13 · 2025: 30 · 2024: 29 · 2023: 24
FWCI
22.53
Type
journal article
Language
en
Open access
Locations
  • Heliyon (publishedVersion, cc-by-nc-nd)
  • PubMed Central (publishedVersion, cc-by-nc-nd)
  • PubMed (publishedVersion)
  • Repositorio Institucional de la Universidad de Burgos (University of Burgos) (submittedVersion, cc-by-nc-nd)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
  • Gobierno de España Ministerio de Ciencia e Innovación (PID2020-117111RB-I00)
  • Ministerio de Ciencia e Innovación
Topics
  • Physical Sciences › Computer Science › Information Systems › Artificial Intelligence in Education
  • Physical Sciences › Computer Science › Artificial Intelligence › Intelligent Tutoring Systems and Adaptive Learning
  • Physical Sciences › Computer Science › Artificial Intelligence › AI in Service Interactions
Keywords
chatbots, student self-regulation, student satisfaction, metacognitive strategies, learning outcomes, prior knowledge, educational attainment, health sciences students

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review