The DOI matches its Crossref record; 23 of the 24 references it cites were found in a registry or an index; 2 of the 11 sources its claims cite were checked for support.
Analysis of the effect of an artificial intelligence chatbot educational program on non-face-to-face classes: a quasi-experimental study
Open accessAbstract
Background: Education and training are needed for nursing students using artificial intelligence-based educational programs. However, few studies have assessed the effect of using chatbots in nursing education. Objectives: This study aimed to develop and examine the effect of an artificial intelligence chatbot educational program for promoting nursing skills related to electronic fetal monitoring in nursing college students during non-face-to-face classes during the COVID-19 pandemic. Design: This quasi-experimental study used a nonequivalent control group non-synchronized pretest–posttest design. Methods: The participants were 61 junior students from a nursing college located in G province of South Korea. Data were collected between November 3 and 16, 2021, and analyzed using independent t-tests. Results: The experimental group—in which the artificial intelligence chatbot program was applied—did not show statistically significant differences in knowledge (t = -0.58, p = .567), clinical reasoning competency (t = 0.75, p = .455), confidence (t = 1.13, p = .264), and feedback satisfaction (t = 1.72, p = .090), compared with the control group; however, its participants’ interest in education (t = 2.38, p = .020) and self-directed learning (t = 2.72, p = .006) were significantly higher than those in the control group. Conclusion: The findings of our study highlighted the potential of artificial intelligence chatbot programs as an educational assistance tool to promote nursing college students’ interest in education and self-directed learning. Moreover, such programs can be effective in enhancing nursing students’ skills in non-face-to face-situations caused by the ongoing COVID-19 pandemic.
Overview
CoChat Summary
TL;DR
This quasi-experimental study developed and tested an AI chatbot educational program for electronic fetal monitoring (EFM) in non-face-to-face nursing classes during COVID-19. The chatbot did not significantly improve knowledge, clinical reasoning competency, confidence, or feedback satisfaction, but it did significantly increase interest in education and self-directed learning.
Key findings
- No statistically significant differences were found between experimental and control groups for EFM knowledge (t = -0.58, p = .567), clinical reasoning competency (t = 0.75, p = .455), confidence in fetal health assessment (t = 1.13, p = .264), or feedback satisfaction (t = 1.72, p = .090).
- The experimental group had significantly higher interest in education than the control group (t = 2.38, p = .020), with mean differences from pre- to post-test of 2.17 ± 2.60 vs. 0.09 ± 1.37.
- The experimental group also had significantly higher self-directed learning than the control group (t = 2.72, p = .006), with mean differences of 2.20 ± 2.43 vs. 0.81 ± 1.47.
- Both groups improved in knowledge from pre- to post-test, but the between-group difference was not significant.
Methods
- Quasi-experimental nonequivalent control group pretest–posttest design; participants were 61 junior nursing students from a nursing college in G province, South Korea, with final groups of 31 control and 30 experimental participants.
- Data were collected November 3–16, 2021; the experimental group attended video lectures plus the AI chatbot program, while the control group attended video lectures only.
- Analysis used SPSS/WIN 23, with independent t-tests for outcome comparisons and chi-squared tests, Fisher’s exact tests, and t-tests for homogeneity testing.
Limitations
- Stated: the study evaluated participants from only one college and used a nonequivalent study design; the authors propose future longitudinal studies to assess medium- or long-term effects.
- Stated: the chatbot was a transaction-type chatbot that provided feedback based on predicted responses but did not address specific student questions, which may have limited feedback satisfaction.
- Evident from the text: the sample was small (N = 61) and data collection occurred over a short two-week period, limiting generalizability and long-term inference.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
2 Valid · 0 Not valid · 5 Not checked · 21 Own results and interpretations
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The experimental group, in which, the AI chatbot educational program for EFM was applied, did not show statistically significant differences in knowledge (t = -0.58, p = 0.567), Clinical reasoning competency (t = 0.75, p = 0.455), confidence in fetal health assessment (t = 1.13, p = 0.264), and feedback satisfaction (t = 1.72, p = 0.090), compared with the control group.”
§ Analysis of the effect of an artificial intelligence chatbot educational program on non-face-to-face classes: a quasi-experimental study > Results > Hypothesis testing
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The experimental group, in which, the AI chatbot educational program for EFM was applied, did not show statistically significant differences in knowledge (t = -0.58, p = 0.567), Clinical reasoning competency (t = 0.75, p = 0.455), confidence in fetal health assessment (t = 1.13, p = 0.264), and feedback satisfaction (t = 1.72, p = 0.090), compared with the control group.”
§ Analysis of the effect of an artificial intelligence chatbot educational program on non-face-to-face classes: a quasi-experimental study > Results > Hypothesis testing
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The experimental group, in which, the AI chatbot educational program for EFM was applied, did not show statistically significant differences in knowledge (t = -0.58, p = 0.567), Clinical reasoning competency (t = 0.75, p = 0.455), confidence in fetal health assessment (t = 1.13, p = 0.264), and feedback satisfaction (t = 1.72, p = 0.090), compared with the control group.”
§ Analysis of the effect of an artificial intelligence chatbot educational program on non-face-to-face classes: a quasi-experimental study > Results > Hypothesis testing
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, participants in the experimental group showed significantly higher interest in education (t = 2.38, p = 0.020) and self-directed learning (t = 2.72, p = 0.006) than those in the control group.”
- Also stated in: § Conclusions · plain
“The chatbot program developed in our study had positive effects on the nursing college students’ interest in education and self-directed learning.”
§ Analysis of the effect of an artificial intelligence chatbot educational program on non-face-to-face classes: a quasi-experimental study > Results > Hypothesis testing
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § Conclusions · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The experimental group, in which, the AI chatbot educational program for EFM was applied, did not show statistically significant differences in knowledge (t = -0.58, p = 0.567), Clinical reasoning competency (t = 0.75, p = 0.455), confidence in fetal health assessment (t = 1.13, p = 0.264), and feedback satisfaction (t = 1.72, p = 0.090), compared with the control group.”
§ Analysis of the effect of an artificial intelligence chatbot educational program on non-face-to-face classes: a quasi-experimental study > Results > Hypothesis testing
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
9 more claims
Create a free account to see all 28 claims and to verify the 1 reference still open.
- It may state two separate claims as one. 13
- The quote only partly states this claim. 1
- Probably not a substantial claim. 1
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“Positive changes in affective learning gains include learning interest (16), self-efficacy (18), satisfaction (25), enjoyment (19), and motivation (11).”
From Digital affordances of AI chatbots in nursing education: a systematic review of learning gains and gaps in the evidence (2026) · cites this paper as [16]
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“and can also be used as a tool of assessment to assess clinical skills (Han et al., 2022)”
From ChatGPT in medicine: an overview of its applications, advantages, limitations, future prospects, and ethical considerations (2023) · cites this paper as Han et al., 2022
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Chatbots, free from time and space constraints, can complement the limitations of simulation education, enabling learners to engage in SDL by facilitating conversations anytime and anywhere [16].”
From Development and effects of a chatbot education program for self-directed learning in nursing students (2025) · cites this paper as [16]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“However, fewer studies (16) have reported significant changes in students’ affective learning gains (e.g., confidence and satisfaction) or cognitive learning gains (e.g., knowledge and clinical reasoning competency).”
From Digital affordances of AI chatbots in nursing education: a systematic review of learning gains and gaps in the evidence (2026) · cites this paper as [16]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In nursing education, AI-enabled tools are increasingly employed to facilitate simulation, knowledge retrieval, feedback provision and self-directed learning, with some reports suggesting enhanced engagement and perceived performance [1,12-15].”
From Artificial Intelligence–Enabled Educational Tools and Clinical Reasoning in Undergraduate Nursing Education: A Systematic Review (2026) · cites this paper as [12]
Experimental — accuracy not yet measured
20 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Rated major: it would touch a result — a value a result rests on, reported and recomputed p-values on opposite sides of the significance level, two passages that conflict with no sign which is right, or a cited work retracted before this paper was published.
In the paper
“| Interest in education | Exp | 6.07 ± 2.23 | 8.40 ± 1.43 | 2.17 ± 2.60 | 2.38 | .020 |”
§ Analysis of the effect of an artificial intelligence chatbot educational program on non-face-to-face classes: a quasi-experimental study > Results > Hypothesis testing
Reported 2.17 · recomputed 2.33
Rated major: it would touch a result — a value a result rests on, reported and recomputed p-values on opposite sides of the significance level, two passages that conflict with no sign which is right, or a cited work retracted before this paper was published.
In the paper
“| Interest in education | Con | 6.87 ± 2.08 | 7.77 ± 1.77 | 0.09 ± 1.37 | 2.38 | .020 |”
§ Analysis of the effect of an artificial intelligence chatbot educational program on non-face-to-face classes: a quasi-experimental study > Results > Hypothesis testing
Reported 0.09 · recomputed 0.9
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
“The participants’ knowledge of EFM was assessed using 13 questions as follows: three questions on understanding and explaining the purpose and method of EFM, seven questions on examining the results of EFM, and three questions on knowledge of nursing interventions based on the EFM results. With one point for every correct answer, the total scores ranged from 0‒20”
§ Analysis of the effect of an artificial intelligence chatbot educational program on non-face-to-face classes: a quasi-experimental study > Methods > Study instruments
Reported 20 · recomputed 13
Citations
Cites
Faul, F., Erdfelder, E., Lang, A.-G., & Buchner, A. (2007) · Behavior Research Methods · doi:10.3758/bf03193146 · Cited by 65,971 · cited 1×
“The sample size was calculated using G*Power version 3.1.9.2 [19].”
MethodInfluentialPredicted Influences of Artificial Intelligence on Nursing Education: Scoping Review
Buchanan, C., Howitt, M. L., Wilson, R., Booth, R. G., Risling, T., & Bamford, M. (2021) · JMIR Nursing · doi:10.2196/23933 · Cited by 326
Shorey, S., Ang, E., Yap, J., Ng, E. D., Lau, S. T., & Chui, C. K. (2019) · Journal of Medical Internet Research · doi:10.2196/14658 · Cited by 238
Chatbot application in a 5th grade science course
Deveci Topal, A., Dilek Eren, C., & Kolburan Geçer, A. (2021) · Education and Information Technologies · doi:10.1007/s10639-021-10627-8 · Cited by 182
Optimizing Students’ Mental Health and Academic Performance: AI-Enhanced Life Crafting
Dekker, I., De Jong, E. M., Schippers, M. C., De Bruijn-Smolders, M., Alexiou, A., & Giesbers, B. (2020) · Frontiers in Psychology · doi:10.3389/fpsyg.2020.01063 · Cited by 176
Liou, S., Liu, H., Tsai, H., Tsai, Y., Lin, Y., Chang, C., & Cheng, C. (2016) · Journal of Advanced Nursing · doi:10.1111/jan.12831 · Cited by 104 · cited 1×
“[22], using a 5-point Likert scale, and translated and validated by Joung and Han [23].”
MethodLeveraging chatbots to improve self-guided learning through conversational quizzes
Pereira, J. (2016) · Ninth International Conference on Technological Ecosystems for Enhancing Multiculturality (TEEM'21) · doi:10.1145/3012430.3012625 · Cited by 83 · cited 2×
“This is similar to the result of a previous study, that used a 15-week chatbot program for students majoring in computer science, which reported that it assisted in self-directed learning since 72% of the students showed improved participation in subjects [24].”
BackgroundResultFryer, L. K., Thompson, A., Nakao, K., Howarth, M., & Gallacher, A. (2020) · Learning and Individual Differences · doi:10.1016/j.lindif.2020.101850 · Cited by 71 · cited 2×
“Since previous studies that assessed the effect of using chatbot in college students’ education had an effect size larger than 0.8 [7, 20], the effect size in this study was set at 0.8, which corresponds to a large effect size, as presented by Cohen [21].”
BackgroundMethodInfluentialQualitative exploration of digital chatbot use in medical education: A pilot study
Kaur, A., Singh, S., Chandan, J. S., Robbins, T., & Patel, V. (2021) · DIGITAL HEALTH · doi:10.1177/20552076211038151 · Cited by 67 · cited 2×
“A chatbot also has various applications in the education field, as people can use it to learn without time and space restrictions [4].”
BackgroundAbbasi, S., & Kazi, H. (2014) · Asian Journal of Applied Science and Engineering · doi:10.15590/ajase/2014/v3i7/53576 · Cited by 67
Research works printed but not linked: 1
- Cohen J. Statistical power analysis for the Behavioral Science. Hillsdale: Erlbaum; 1988.
Cited by
Dave, T., Athaluri, S. A., & Singh, S. (2023) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2023.1169595 · Cited by 1,204
“…medical students, doctors, nurses, and all members of the healthcare fraternity alike to know about updates and new developments in their respective fields, and can also be used as a tool of assessment to assess clinical skills (Han et al., 2022) thus playing a pivotal role in medical education.”
MethodDo AI chatbots improve students learning outcomes? Evidence from a meta‐analysis
Wu, R., & Yu, Z. (2024) · British Journal of Educational Technology · doi:10.1111/bjet.13334 · Cited by 576
Roles, Users, Benefits, and Limitations of Chatbots in Health Care: Rapid Review (Preprint)
Laymouna, M., Ma, Y., Lessard, D., Schuster, T., Engler, K., & Lebouché, B. (2024) · doi:10.2196/preprints.56930 · Cited by 302
Deng, X., & Yu, Z. (2023) · Sustainability · doi:10.3390/su15042940 · Cited by 298
“With chatbot technology, students experienced more learning interest than those without it, supporting previous studies [10,43,46].”
BackgroundResultRoles, Users, Benefits, and Limitations of Chatbots in Health Care: Rapid Review
Laymouna, M., Ma, Y., Lessard, D., Schuster, T., Engler, K., & Lebouché, B. (2024) · Journal of Medical Internet Research · doi:10.2196/56930 · Cited by 305
Metadata
- Authors
- Han, J.-W., Park, J., & Lee, H.
- Affiliations
- Kyung Hee University (KR)
- Gangneung–Wonju National University (KR)
- Corresponding author
- Lee, Hanna
- Venue
- BMC Medical Education
- Published
- 2022-12-01
- Volume
- 22
- Issue
- 1
- Pages
- 830-830
- ISSN
- 1472-6920 (ISSN-L)
- PMC
- PMC9713176
- OpenAlex
- W4310494163
- Cited by
- 187
- References
- 24
- Citations by year
- 2026: 52 · 2025: 51 · 2024: 60 · 2023: 24
- FWCI
- 10.18
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Locations
- BMC Medical Education (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- National Research Foundation of Kore (2022R1I1A3072862)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Social Sciences › Education › Artificial Intelligence in Education
- Physical Sciences › Computer Science › Artificial Intelligence › Intelligent Tutoring Systems and Adaptive Learning
- Keywords
- chatbots, artificial intelligence in education, nursing education, fetal heart rate monitoring, non-face-to-face teaching, COVID-19 pandemic, clinical reasoning, self-directed learning, student self-confidence, nursing students
- MeSH terms
- COVID-19, Artificial Intelligence, Humans, Learning, Students, Nursing, Pandemics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Analysis of the effect of an artificial intelligence chatbot educational program on non-face-to-face classes: a quasi-experimental study. https://app.cochat.ai/papers/10.1186/s12909-022-03898-3
- The paper
- Han, J.-W., Park, J., & Lee, H. (2022). Analysis of the effect of an artificial intelligence chatbot educational program on non-face-to-face classes: a quasi-experimental study. BMC Medical Education. https://doi.org/10.1186/s12909-022-03898-3
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.

