CoChat
Sign Up
Cited by 50Artificial Intelligence in Healthcare and EducationInnovations in Medical EducationArtificial Intelligence in Education

Health Care Trainees’ and Professionals’ Perceptions of ChatGPT in Improving Medical Knowledge Training: Rapid Survey Study

Open access
Hu, Je-Ming, Liu, Feng-Cheng, Chu, Chi-Ming, Chang, Yu-Tien·2023·Journal of Medical Internet Research
DOI 10.2196/49385PMC PMC10620632

Published version of a preprint

Open this paper in CoChat — free accountOpen PDF

Abstract

Background: ChatGPT is a powerful pretrained large language model. It has both demonstrated potential and raised concerns related to knowledge translation and knowledge transfer. To apply and improve knowledge transfer in the real world, it is essential to assess the perceptions and acceptance of the users of ChatGPT-assisted training. Objective: We aimed to investigate the perceptions of health care trainees and professionals on ChatGPT-assisted training, using biomedical informatics as an example. Methods: We used purposeful sampling to include all health care undergraduate trainees and graduate professionals (n=195) from January to May 2023 in the School of Public Health at the National Defense Medical Center in Taiwan. Subjects were asked to watch a 2-minute video introducing 5 scenarios about ChatGPT-assisted training in biomedical informatics and then answer a self-designed online (web- and mobile-based) questionnaire according to the Kirkpatrick model. The survey responses were used to develop 4 constructs: “perceived knowledge acquisition,” “perceived training motivation,” “perceived training satisfaction,” and “perceived training effectiveness.” The study used structural equation modeling (SEM) to evaluate and test the structural model and hypotheses. Results: The online questionnaire response rate was 152 of 195 (78%); 88 of 152 participants (58%) were undergraduate trainees and 90 of 152 participants (59%) were women. The ages ranged from 18 to 53 years (mean 23.3, SD 6.0 years). There was no statistical difference in perceptions of training evaluation between men and women. Most participants were enthusiastic about the ChatGPT-assisted training, while the graduate professionals were more enthusiastic than undergraduate trainees. Nevertheless, some concerns were raised about potential cheating on training assessment. The average scores for knowledge acquisition, training motivation, training satisfaction, and training effectiveness were 3.84 (SD 0.80), 3.76 (SD 0.93), 3.75 (SD 0.87), and 3.72 (SD 0.91), respectively (Likert scale 1-5: strongly disagree to strongly agree). Knowledge acquisition had the highest score and training effectiveness the lowest. In the SEM results, training effectiveness was influenced predominantly by knowledge acquisition and partially met the hypotheses in the research framework. Knowledge acquisition had a direct effect on training effectiveness, training satisfaction, and training motivation, with β coefficients of .80, .87, and .97, respectively (all P<.001). Conclusions: Most health care trainees and professionals perceived ChatGPT-assisted training as an aid in knowledge transfer. However, to improve training effectiveness, it should be combined with empirical experts for proper guidance and dual interaction. In a future study, we recommend using a larger sample size for evaluation of internet-connected large language models in medical knowledge transfer.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.2196/49385

Cites

37 of 37 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Eight studies examined the knowledge, attitudes, and skills of medical students and educators regarding LLMs, specifically ChatGPT; students generally expressed a more positive attitude toward LLMs than practicing physicians; those more knowledgeable about LLMs tended to hold a more positive attitude and were more likely to use them for educational purposes; and concerns were raised regarding the

    “Eight studies examined the knowledge, attitudes, and skills of medical students and educators regarding LLMs, specifically ChatGPT [24, 26–29, 79–81]. Students generally expressed a more positive attitude toward LLMs than practicing physicians, showing optimism about their potential to support learners and educators [81]. Those who were more knowledgeable about LLMs tended to hold a more positive”

    From Artificial intelligence in undergraduate medical education: an updated scoping review (2025) · cites this paper as [28]

    Split into 2 atoms — 1 confirmed, 0 partially supported, 1 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.

    Checked by a reasoning model — may be wrong.

    Experimental — accuracy not yet measured

  • These studies involved faculty and mostly medical students, but also other health-related students from countries worldwide, including Europe, the Middle East, northern and sub-Saharan Africa, North America, Asia, Australia, and New Zealand.

    “These studies involved faculty and mostly medical students, but also other health-related students from countries worldwide, including Europe, the Middle East, northern and sub-Saharan Africa, North America, Asia, Australia, and New Zealand [24–29].”

    From Artificial intelligence in undergraduate medical education: an updated scoping review (2025) · cites this paper as [28]

    Split into 2 atoms — 0 confirmed, 2 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.

    Experimental — accuracy not yet measured

  • Although the number of respondents exceeded the required sample size, the response rate was lower compared with similar studies, which may limit the generalizability of the findings.

    “First, although the number of respondents exceeded the required sample size, the response rate was lower compared with similar studies [19, 21, 22, 24, 25, 29, 31, 34, 37, 40, 62], which may limit the generalizability of the findings.”

    From Knowledge and use, Perceptions of Benefits and Limitations of Artificial Intelligence Chatbots among Italian Physiotherapy students: a cross-sectional National Study (2025) · cites this paper as [22]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 2

  1. [14]White J Fu Q Hays S Sandborn M Olea C Gilbert H A prompt pattern catalog to enhance prompt engineering with ChatGPT arXiv Preprint posted online Feb 21, 2023.
  2. [26]Brown T Mann B Ryder N Subbiah M Kaplan J Dhariwal P Language models are few-shot learners Advances in Neural Information Processing Systems 2020 NeurIPS 2020 Dec 6-12, 2020 Virtual 33 901

Cited by

Sorted by citations

Related

Authors
Hu, J.-M., Liu, F.-C., Chu, C.-M., & Chang, Y.-T.
Affiliations
  • Tri-Service General Hospital (TW)
  • Fu Jen Catholic University (TW)
  • National Defense Medical Center (TW)
Corresponding author
Chang, Yu-Tien
ORCID
Venue
Journal of Medical Internet Research
Publisher
JMIR Publications Inc.
Published
2023-10-18
Volume
25
Pages
e49385
ISSN
1438-8871 (ISSN-L)
DOI
10.2196/49385
PMC
PMC10620632
OpenAlex
W4387728656
Cited by
50
References
37
Citations by year
2026: 12 · 2025: 18 · 2024: 18 · 2023: 2
FWCI
2.00
Type
journal article
Language
en
Open access
Licence
cc-by
Locations
  • Journal of Medical Internet Research (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Topics
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Innovations in Medical Education
  • Physical Sciences › Computer Science › Information Systems › Artificial Intelligence in Education
Keywords
ChatGPT, large language models, medical education, professional perceptions, biomedical informatics, knowledge acquisition, training effectiveness, training satisfaction, training motivation, Kirkpatrick model, artificial intelligence in education, public health training, knowledge transfer
MeSH terms
Adolescent, Adult, Artificial Intelligence, Attitude of Health Personnel, Female, Humans, Male, Medicine, Middle Aged, Perception, Surveys and Questionnaires, Students, Taiwan, Young Adult

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review