Fact Check: Assessing the Response of ChatGPT to Alzheimer's Disease Myths
Open accessAbstract
Introduction There are many myths regarding Alzheimer's disease (AD) that have been circulated on the internet, each exhibiting varying degrees of accuracy, inaccuracy, and misinformation. Large language models, such as ChatGPT, may be a valuable tool to help assess these myths for veracity and inaccuracy; however, they can induce misinformation as well. Objective This study assesses ChatGPT's ability to identify and address AD myths with reliable information. Methods We conducted a cross-sectional study of attending geriatric medicine clinicians' evaluation of ChatGPT (GPT 4.0) responses to 16 selected AD myths. We prompted ChatGPT to express its opinion on each myth and implemented a survey using REDCap to determine the degree to which clinicians agreed with the accuracy of each of ChatGPT's explanations. We also collected their explanations of any disagreements with ChatGPT's responses. We used a 5-category Likert-type scale with a score ranging from -2 to 2 to quantify clinicians' agreement in each aspect of the evaluation. Results The clinicians (n = 10) were generally satisfied with ChatGPT's explanations. Among the 16 myths, the clinicians were generally satisfied with these explanations, with [mean (SD) score of 1.1(±0.3)]. Most clinicians selected "Agree" or "Strongly Agree" for each statement. Some statements obtained a small number of "Disagree" responses. There were no "Strongly Disagree" responses. Conclusion Most surveyed health care professionals acknowledged t…
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 48,293 · see citing works
The REDCap consortium: building an international community of software platform partners
Harris, P. A., Taylor, R., Minor, B. L., Elliott, V., Fernandez, M., O'Neal, L., McLeod, L., Delacqua, G., Delacqua, F., Kirby, J., et al. (2019) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2019.103208 · Cited by 24,013 · see citing works
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929 · see citing works
Ayers, J. W., Poliak, A., Dredze, M., Leas, E. C., Zhu, Z., Kelley, J. B., Faix, D. J., Goodman, A. M., Longhurst, C. A., Hogarth, M., et al. (2023) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2023.1838 · Cited by 2,613 · see citing works
Unified Language Model Pre-training for Natural Language Understanding and Generation
Dong, L., Yang, N., Wang, W., Wei, F., Liu, X., Wang, Y., Gao, J., Zhou, M., & Hon, H.-W. (2019) · arXiv · doi:10.48550/arxiv.1905.03197 · Cited by 1,225 · see citing works
Johnson, S. B., King, A. J., Warner, E. L., Aneja, S., Kann, B. H., & Bylund, C. L. (2023) · JNCI Cancer Spectrum · doi:10.1093/jncics/pkad015 · Cited by 289 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Urban, M., Brom, C., Lukavský, J., Děchtěrenko, F., Hein, V., Svacha, F., Kmoníčková, P., & Urban, K. (2025) · British Journal of Educational Technology · doi:10.1111/bjet.13591 · Cited by 47 · see citing works
- Authors
- Huang, S. S., Song, Q., Beiting, K. J., Duggan, M. C., Hines, K., Murff, H., Leung, V., Powers, J., Harvey, T. S., Malin, B., & Yin, Z.
- Affiliations
- Vanderbilt University Medical Center (US)
- Vanderbilt University (US)
- VA Tennessee Valley Healthcare System (US)
- University of Illinois Chicago (US)
- Corresponding author
- Huang, Sean S.
- ORCID
- Venue
- Journal of the American Medical Directors Association
- Publisher
- Elsevier BV
- Published
- 2024-10
- Volume
- 25
- Issue
- 10
- Pages
- 105178
- ISSN
- 1525-8610 (ISSN-L)
- OpenAlex
- W4401287356
- Cited by
- 9
- References
- 6
- Citations by year
- 2026: 2 · 2025: 5 · 2024: 1 · 2023: 1
- FWCI
- 0.74
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Journal of the American Medical Directors Association (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Funders
- National Institutes of Health
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
- Health Sciences › Medicine › Psychiatry and Mental health › Dementia and Cognitive Impairment Research
- Keywords
- Alzheimer's disease, health myths, misinformation, ChatGPT, large language models, elderly care
- MeSH terms
- Alzheimer Disease, Communication, Cross-Sectional Studies, Female, Humans, Male, Surveys and Questionnaires, Internet
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
