A mixed methods crossover randomized controlled trial exploring the experiences, perceptions, and usability of artificial intelligence (ChatGPT) in health sciences education
Open accessAbstract
Background: Generative artificial intelligence (AI) integrated programs such as Chat Generative Pre-trained Transformers (ChatGPT) are becoming more widespread in educational settings, with mounting ethical and reliability concerns regarding its usage. This paper explores the experiences, perceptions, and usability of ChatGPT in undergraduate health sciences students. Methods: Twenty-seven students at Carleton University (Canada) were enrolled in a crossover randomized controlled trial study from a Health Sciences course during the Fall 2023 academic term. The intervention condition involved the use of ChatGPT-3.5, whereas the control condition involved using conventional web-based tools. Technology usability was compared between ChatGPT-3.5 and the traditional tools using questionnaires. Focus group discussions were conducted with seven students to further elaborate on student perceptions and experiences. Reflexive thematic analysis was employed to identify themes from the focus group data. Results: Easiness of learnability for personal use and a perception of quick learnability towards ChatGPT-3.5 were significantly higher, compared to conventional online tools from the Systems Usability Scale. Qualitative results highlighted strong benefits of ChatGPT-3.5, such as being a tool for increased overall productivity and brainstorming. However, students identified challenges associated with reliability and accuracy, and concerns about academic integrity. Conclusions: Despite the benefits and positive usability of ChatGPT-3.5 identified by students, an explicit need for the development of policies, procedures and regulations remains. An established framework of best practices for the usage of AI within health science education is necessary. This will ensure accountability of users and lead to a more effective integration of AI technologies into academic settings.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Health science students found ChatGPT-3.5 easy to learn but noted its shortcomings in reliability, accuracy and academic integrity.
“Additionally, in a focus group study, health science students found ChatGPT- 3.5 easy to learn but noted its shortcomings in reliability, accuracy and academic integrity [24].”
From Generative artificial intelligence in physiotherapy education: great potential amidst challenges- a qualitative interview study (2025) · cites this paper as [24]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
The citation lists haven’t been fetched yet
References 27 · Cited by 25
- Authors
- Veras, M., Dyer, J.-O., Shannon, H., Bogie, B. J. M., Ronney, M., Sekhon, H., Rutherford, D., Silva, P. G. B., & Kairy, D.
- Affiliations
- Department of Physical Therapy, College of Rehabilitation Sciences, University of Manitoba, Winnipeg, MB, Canada
- Université de Montréal (CA)
- Carleton University (CA)
- University of Ottawa (CA)
- Teaching and Learning Services, Carleton University, Ottawa, ON, Canada
- Centre for Addiction and Mental Health (CA)
- Dalhousie University (CA)
- Centro Universitário Christus, Fortaleza, Ceará, Brazil
- ORCID
- Venue
- DIGITAL HEALTH
- Publisher
- SAGE Publications
- Published
- 2024-09
- Volume
- 10
- Pages
- 20552076241298485-20552076241298485
- ISSN
- 2055-2076 (ISSN-L)
- PMC
- PMC11800248
- OpenAlex
- W4405123909
- Cited by
- 25
- References
- 27
- Citations by year
- 2026: 13 · 2025: 12
- FWCI
- 2.34
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Digital Health (publishedVersion, cc-by-nc)
- PubMed Central (publishedVersion, cc-by-nc)
- PubMed (publishedVersion)
- Carleton University's Institutional Repository (MacOdrum Library, Carleton University) (submittedVersion, cc-by-nc-nd)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Scholarship of Teaching and Learning Grant from Carleton University
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Information Systems › Artificial Intelligence in Education
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Innovations in Medical Education
- Keywords
- ChatGPT, generative AI, health sciences education, technology usability, student perceptions, academic integrity, artificial intelligence in higher education, AI policy and regulation, undergraduate students
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
