CoChat

Using ChatGPT for medical education: the technical perspective

Open access
Chan, Kwan Yin, Yuen, Tsz Hon, Co, Michael · 2025 · BMC Medical Education

The DOI matches its Crossref record; 77 of the 77 references it cites were found in a registry or an index.

Abstract

BACKGROUND: The chatbot application Bennie and the Chats was introduced due to the outbreak of COVID-19, which is aimed to provide substitution for teaching conventional clinical history-taking skills. It was implemented with DialogFlow with preset responses, which consists of a large constraint on responding to different conversations. The rapid advancement of artificial intelligence, such as the recent introduction of ChatGPT, offers innovative conversational experiences with computer-generated responses. It provides an idea to develop the second generation of Bennie and the Chats. As the epidemic slows, it can become an assisting tool for students as additional exercise. In this work, we present the second generation of Bennie and the Chats with ChatGPT, which provides room for flexible and expandable improvement. METHODS: The objective of this research is to examine the influence of the newly proposed chatbot on learning efficacy and experiences in bedside teaching, and its potential contributions to international teaching collaboration. This study employs a mixed-method design that incorporates both quantitative and qualitative approaches. From the quantitative approach, we launched the world's first cross-territory virtual bedside teaching with our proposed application and conducted a survey between the University of Hong Kong (HKU) and the National University of Singapore (NUS). Descriptive statistics and Spearman's Correlation were applied for data analysis. From the qualitative approach, a comparative analysis was conducted between the two versions of the chatbot. And, we discuss the interrelationship between the quantitative and qualitative results. RESULTS: For the quantitative result, we collected a questionnaire from 45 students about the evaluation of virtual bedside teaching between territories. Over 75% of the students agreed that teaching can enhance learning effectiveness and experience. Moreover, by exchanging patients cases, 82.2% of students agreed that it helps to gain more experiences with diseases that may not be prevalent in their own locality. For the qualitative result, the new chatbot provides better usability and flexibility. CONCLUSION: Virtual bedside teaching with chatbots has revolutionized conventional bedside teaching by its advantages and allowing international collaborations. We believe that the training of history taking skills by chatbot will be a feasible supplementary teaching tool to conventional bedside teaching.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1186/s12909-025-06785-9

Cites

77 of 77 references verified0 citation errors · 0 not checkable

Statistics

Figures

Fig. 1 The hierarchy overview of the application
Fig. 1 The hierarchy overview of the applicationCC BY-NC-ND · PubMed Central
Fig. 2 The structure overview of the application
Fig. 2 The structure overview of the applicationCC BY-NC-ND · PubMed Central
Fig. 3 An example of a JSON data entry. We hide the details of the prompt
Fig. 3 An example of a JSON data entry. We hide the details of the promptCC BY-NC-ND · PubMed Central
Fig. 4 The login page (left), the homepage of administrators (center) and users (right)
Fig. 4 The login page (left), the homepage of administrators (center) and users (right)CC BY-NC-ND · PubMed Central
Fig. 5 (From left to right) The pages of selection among divisions, wards, cases, and the chatroom
Fig. 5 (From left to right) The pages of selection among divisions, wards, cases, and the chatroomCC BY-NC-ND · PubMed Central
Fig. 6 Examples of responses from ChatGPT under different personalities: (from left to right) polite, rude
Fig. 6 Examples of responses from ChatGPT under different personalities: (from left to right) polite, rudeCC BY-NC-ND · PubMed Central
Fig. 7 Examples of responses from ChatGPT under different personalities: (from left to right) talkative, grumpy
Fig. 7 Examples of responses from ChatGPT under different personalities: (from left to right) talkative, grumpyCC BY-NC-ND · PubMed Central
Fig. 8 The examples of unexpected behaviour from ChatGPT after receiving inappropriate language in the conversation
Fig. 8 The examples of unexpected behaviour from ChatGPT after receiving inappropriate language in the conversationCC BY-NC-ND · PubMed Central
Fig. 9 Frequency distribution of the overall ratings
Fig. 9 Frequency distribution of the overall ratingsCC BY-NC-ND · PubMed Central
Fig. 10 Frequency distribution of the rating on “user-friendliness", “identification", and “interaction". NR means no response
Fig. 10 Frequency distribution of the rating on “user-friendliness", “identification", and “interaction". NR means no responseCC BY-NC-ND · PubMed Central
Fig. 11 Scatter diagrams of the rating between “user-friendliness", “identification", “interaction" and “efficacy"
Fig. 11 Scatter diagrams of the rating between “user-friendliness", “identification", “interaction" and “efficacy"CC BY-NC-ND · PubMed Central
Fig. 12 Scatter diagrams of the rating between “user-friendliness", “identification", “interaction" and “experience"
Fig. 12 Scatter diagrams of the rating between “user-friendliness", “identification", “interaction" and “experience"CC BY-NC-ND · PubMed Central
Fig. 13 The handling of irrelevant questions: Dialogflow (left) and ChatGPT (right)
Fig. 13 The handling of irrelevant questions: Dialogflow (left) and ChatGPT (right)CC BY-NC-ND · PubMed Central
Fig. 14 An example of conversation under English (left) and traditional Chinese (right) with a prescribed patient
Fig. 14 An example of conversation under English (left) and traditional Chinese (right) with a prescribed patientCC BY-NC-ND · PubMed Central
Fig. 15 An example of conversation in Cantonese and occasional English words with a prescribed patient
Fig. 15 An example of conversation in Cantonese and occasional English words with a prescribed patientCC BY-NC-ND · PubMed Central
Fig. 16 An example of education level characteristics: secondary school level (left) and university level (right)
Fig. 16 An example of education level characteristics: secondary school level (left) and university level (right)CC BY-NC-ND · PubMed Central
Fig. 17 Examples of AI medical images generated by DALL·E 2
Fig. 17 Examples of AI medical images generated by DALL·E 2CC BY-NC-ND · PubMed Central
Fig. 18 An example of AI-generated photos of patients. The eyes of the patients are intentionally concealed
Fig. 18 An example of AI-generated photos of patients. The eyes of the patients are intentionally concealedCC BY-NC-ND · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 77 references

Research works printed but not linked: 6

  1. Swan K. Learning effectiveness: What the research tells us. Elem Qual Online Educ Pract Direction. 2003;4:13–47.
  2. Nguyen T. The effectiveness of online learning: Beyond no significant difference and future horizons. MERLOT J Online Learn Teach. 2015;11(2):309–19.
  3. Loh CYR, Teo TC. Understanding Asian students learning styles, cultural influence and learning strategies. J Educ Soc Policy. 2017;7(1):194–210.
  4. Radford A, Wu J, Child R, Luan D, Amodei D, Sutskever I, et al. Language models are unsupervised multitask learners. OpenAI Blog. 2019;1(8):9.
  5. Christiano PF, Leike J, Brown T, Martic M, Legg S, Amodei D. Deep reinforcement learning from human preferences. Adv Neural Inf Process Syst. 2017;30:4302–10.
  6. GDPR. General data protection regulation. Regulation (EU). 2016;679:1–88.

Cited by

See all 38

Metadata

Authors
Chan, K. Y., Yuen, T. H., & Co, M.
Affiliations
  • University of Hong Kong (HK)
  • Monash University (AU)
  • Queen Mary Hospital (CN)
Corresponding author
Chan, Kwan Yin
Venue
BMC Medical Education
Published
2025-02-07
Volume
25
Issue
1
Pages
201-201
ISSN
1472-6920 (ISSN-L)
OpenAlex
W4407240412
Cited by
34
References
74
Citations by year
2026: 20 · 2025: 14
FWCI
8.11
Type
journal article
Language
en
Open access
Licence
Locations
  • BMC Medical Education (publishedVersion, cc-by-nc-nd)
  • PubMed Central (publishedVersion, cc-by-nc-nd)
  • PubMed (publishedVersion)
  • Monash University Research Portal (Monash University) (submittedVersion, cc-by-nc-nd)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
  • The HKU Scholars Hub (University of Hong Kong) (submittedVersion, cc-by-nc-nd)
Funders
Research Grant Council of Hong Kong (106220117)
Topics
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Social Sciences › Education › Innovations in Medical Education
  • Social Sciences › Education › Artificial Intelligence in Education
Keywords
ChatGPT, chatbots, medical education, Dialogflow, student perceptions, medical students
MeSH terms
COVID-19, SARS-CoV-2, Generative Artificial Intelligence, Artificial Intelligence, Computer-Assisted Instruction, Education, Medical, Hong Kong, Humans, Singapore

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Using ChatGPT for medical education: the technical perspective. https://app.cochat.ai/papers/10.1186/s12909-025-06785-9
The paper
Chan, K. Y., Yuen, T. H., & Co, M. (2025). Using ChatGPT for medical education: the technical perspective. BMC Medical Education. https://doi.org/10.1186/s12909-025-06785-9

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.