Predicting dementia from spontaneous speech using large language models
Open accessThe DOI matches its Crossref record; 39 of the 39 references it cites were found in a registry or an index.
Abstract
Language impairment is an important biomarker of neurodegenerative disorders such as Alzheimer’s disease (AD). Artificial intelligence (AI), particularly natural language processing (NLP), has recently been increasingly used for early prediction of AD through speech. Yet, relatively few studies exist on using large language models, especially GPT-3, to aid in the early diagnosis of dementia. In this work, we show for the first time that GPT-3 can be utilized to predict dementia from spontaneous speech. Specifically, we leverage the vast semantic knowledge encoded in the GPT-3 model to generate text embedding, a vector representation of the transcribed text from speech, that captures the semantic meaning of the input. We demonstrate that the text embedding can be reliably used to (1) distinguish individuals with AD from healthy controls, and (2) infer the subject’s cognitive testing score, both solely based on speech data. We further show that text embedding considerably outperforms the conventional acoustic feature-based approach and even performs competitively with prevailing fine-tuned models. Together, our results suggest that GPT-3 based text embedding is a viable approach for AD assessment directly from speech and has the potential to improve early diagnosis of dementia.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures


Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Even in dementia syndromes where language symptoms are not dominant, such as the behavioral variant of frontotemporal dementia, automated speech analysis can detect subtle linguistic and acoustic patterns that reflect underlying cognitive, behavioral, or socio-emotional changes.91,92”
From Review of Artificial Intelligence for Clinical Use in Alzheimer's Disease and Related Dementias (2026) · cites this paper as [91]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Folstein, M. F., Folstein, S. E., & McHugh, P. R. (1975) · Journal of Psychiatric Research · doi:10.1016/0022-3956(75)90026-6 · Cited by 88,282 · cited 12×
“MMSE score prediction.”
BackgroundMethodInfluentialBERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173 · cited 1×
“The current NLP landscape has been revolutionized by large language models [23,26].”
BackgroundScikit-learn: Machine Learning in Python
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Müller, A., Nothman, J., Louppe, G., et al. (2012) · arXiv · doi:10.48550/arxiv.1201.0490 · Cited by 71,016 · cited 2×
“We use the scikit-learn library for the implementation of these models [42].”
MethodGlove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071
Efficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568
Hirono, N., Mori, E., Ikejiri, Y., Imamura, T., Shimomura, T., Hashimoto, M., Yamashita, H., & Ikeda, M. (1997) · Brain and nerve · Cited by 21,038
Kok, J. S., Berg, I. J., & Scherder, E. J. A. (2013) · Dementia and Geriatric Cognitive Disorders Extra · doi:10.1159/000353441 · Cited by 20,589
Older Adults' Reasons for Using Technology while Aging in Place
Peek, S. T. M., Luijkx, K. G., Rijnaard, M. D., Nieboer, M. E., van der Voort, C. S., Aarts, S., van Hoof, J., Vrijhoef, H. J. M., & Wouters, E. J. M. (2016) · Gerontology · doi:10.1159/000430949 · Cited by 19,690
Sensitivity and False Alarm Rate of a Fall Sensor in Long-Term Fall Detection in the Elderly
Kangas, M., Korpelainen, R., Vikman, I., Nyberg, L., & Jämsä, T. (2015) · Gerontology · doi:10.1159/000362720 · Cited by 18,909
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929 · cited 1×
“Fine-tuning has become the de facto standard to leverage large pretrained models to perform downstream tasks [24,25,32].”
BackgroundTransformers: State-of-the-Art Natural Language Processing
Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., et al. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations · doi:10.18653/v1/2020.emnlp-demos.6 · Cited by 8,741
librosa: Audio and Music Signal Analysis in Python
McFee, B., Raffel, C., Liang, D., Ellis, D., McVicar, M., Battenberg, E., & Nieto, O. (2015) · Proceedings of the Python in Science Conference · doi:10.25080/majora-7b98e3ed-003 · Cited by 3,033
Opensmile: the munich versatile and fast open-source audio feature extractor
Eyben, F., Wöllmer, M., & Schuller, B. (2010) · Proceedings of the 18th ACM international conference on Multimedia · doi:10.1145/1873951.1874246 · Cited by 2,642 · cited 1×
“In this work, acoustic features are extracted directly from speech using OpenSMILE (open-source Speech and Music Interpretation by Large-space Extraction), a widely used open-source toolkit for audio feature extraction and classification of speech and music signals [39].”
Methodwav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations
Baevski, A., Zhou, H., Mohamed, A., & Auli, M. (2020) · arXiv · doi:10.48550/arxiv.2006.11477 · Cited by 2,369 · cited 1×
“0 pretrained model [36], a state-of-the-art model for automatic speech recognition.”
Background2021 Alzheimer's disease facts and figures
(2021) · Alzheimer's & Dementia · doi:10.1002/alz.12328 · Cited by 2,201
CNN architectures for large-scale audio classification
Hershey, S., Chaudhuri, S., Ellis, D. P. W., Gemmeke, J. F., Jansen, A., Moore, R. C., Plakal, M., Platt, D., Saurous, R. A., Seybold, B., et al. (2017) · 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2017.7952132 · Cited by 2,023
The Geneva Minimalistic Acoustic Parameter Set (GeMAPS) for Voice Research and Affective Computing
Eyben, F., Scherer, K. R., Schuller, B. W., Sundberg, J., Andre, E., Busso, C., Devillers, L. Y., Epps, J., Laukka, P., Narayanan, S. S., et al. (2016) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2015.2457417 · Cited by 1,818 · cited 1×
“We primarily used the extended Geneva Minimalistic Acoustic Parameter Set (eGeMAPS) features due to their potential to detect physiological changes in voice production, as well as theoretical significance and proven usefulness in previous studies [40].”
MethodCurrent and Future Treatments in Alzheimer Disease: An Update
Yiannopoulou, K. G., & Papageorgiou, S. G. (2020) · Journal of Central Nervous System Disease · doi:10.1177/1179573520907397 · Cited by 966 · cited 1×
“Given its prevalence and still no cure available for AD treatment [2], there is an urgent need for the early diagnosis of dementia, which would yield clear benefits in improving quality of life for individuals with dementia.”
BackgroundLinguistic Features Identify Alzheimer’s Disease in Narrative Speech
Fraser, K. C., Meltzer, J. A., & Rudzicz, F. (2015) · Journal of Alzheimer's Disease · doi:10.3233/jad-150520 · Cited by 797
The Natural History of Alzheimer's Disease: Description of Study Cohort and Accuracy of Diagnosis
Becker, J. T. (1994) · Archives of Neurology · doi:10.1001/archneur.1994.00540180063015 · Cited by 576
Automatic speech analysis for the assessment of patients with predementia and Alzheimer's disease
König, A., Satt, A., Sorin, A., Hoory, R., Toledo‐Ronen, O., Derreumaux, A., Manera, V., Verhey, F., Aalten, P., Robert, P. H., et al. (2015) · Alzheimer's & Dementia: Diagnosis, Assessment & Disease Monitoring · doi:10.1016/j.dadm.2014.11.012 · Cited by 380
García, S. D. L. F., Ritchie, C. W., & Luz, S. F. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.06047 · Cited by 265
Detecting Cognitive Decline Using Speech Only: The ADReSSo Challenge
Luz, S. F., Haider, F., de la Fuente, S., Fromm, D., & MacWhinney, B. (2021) · Interspeech (USB) · doi:10.21437/interspeech.2021-1220 · Cited by 218
Linguistic markers predict onset of Alzheimer's disease
Eyigoz, E., Mathur, S., Santamaria, M., Cecchi, G., & Naylor, M. (2020) · EClinicalMedicine · doi:10.1016/j.eclinm.2020.100583 · Cited by 206
Haider, F., de la Fuente, S., & Luz, S. (2020) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2019.2955022 · Cited by 190
Artificial intelligence for brain diseases: A systematic review
Segato, A., Marzullo, A., Calimeri, F., & De Momi, E. (2020) · APL Bioengineering · doi:10.1063/5.0011697 · Cited by 169 · cited 1×
“It is preferred to develop ethical and legal systems for the implementation, validation and control of AI in clinical care [34].”
BackgroundPredicting probable Alzheimer's disease using linguistic deficits and biomarkers
Orimaye, S. O., Wong, J. S.-M., Golden, K. J., Wong, C. P., & Soyiri, I. N. (2017) · BMC Bioinformatics · doi:10.1186/s12859-016-1456-0 · Cited by 158
Text and Code Embeddings by Contrastive Pre-Training
Neelakantan, A., Xu, T., Puri, R., Radford, A., Han, J. M., Tworek, J., Yuan, Q., Tezak, N., Kim, J. W., Hallacy, C., et al. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2201.10005 · Cited by 145
Gupta, Y., Lee, K. H., Choi, K. Y., Lee, J. J., Kim, B. C., Kwon, G. R., the National Research Center for Dementia, & Initiative, A. D. N. (2019) · PLOS ONE · doi:10.1371/journal.pone.0222446 · Cited by 107 · cited 1×
“, Mini-Mental State Examination (MMSE) [3] for evaluating the progression of AD [4,5].”
MethodA Review of Automated Speech and Language Features for Assessment of Cognitive and Thought Disorders
Voleti, R., Liss, J. M., & Berisha, V. (2020) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2019.2952087 · Cited by 96 · cited 1×
“Previous works on speech analysis are mainly based on the feature-based approach using acoustic features extracted from the speech audio and the linguistic features derived from the written texts or speech transcripts through NLP techniques [7].”
MethodAmini, S., Hao, B., Zhang, L., Song, M., Gupta, A., Karjadi, C., Kolachalama, V. B., Au, R., & Paschalidis, I. C. (2023) · Alzheimer's & Dementia · doi:10.1002/alz.12721 · Cited by 76
Chan, C. C., Fage, B. A., Burton, J. K., Smailagic, N., Gill, S. S., Herrmann, N., Nikolaou, V., Quinn, T. J., Noel-Storr, A. H., & Seitz, D. P. (2019) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd011414.pub2 · Cited by 73
Seitz, D. P., Chan, C. C., Newton, H. T., Gill, S. S., Herrmann, N., Smailagic, N., Nikolaou, V., & Fage, B. A. (2018) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd011415.pub2 · Cited by 70
Yamada, Y., Shinkawa, K., Kobayashi, M., Caggiano, V., Nemoto, M., Nemoto, K., & Arai, T. (2021) · Journal of Alzheimer's Disease · doi:10.3233/jad-210684 · Cited by 56 · cited 1×
“Leveraging AI with the growing development of largescale, multi-modal data such as neuroimaging, speech and language, behavioral biomarkers, and patient information on electronic medical records, will help alleviate the data problem and allow for more accurate, efficient, and early diagnosis [33].”
BackgroundPauses for Detection of Alzheimer’s Disease
Yuan, J., Cai, X., Bian, Y., Ye, Z., & Church, K. (2021) · Frontiers in Computer Science · doi:10.3389/fcomp.2020.624488 · Cited by 49 · cited 1×
“Recent work has shown encouraging results with fine-tuned BERT for AD detection [20,41].”
ResultExploring neural models for predicting dementia from language
Kong, W., Jang, H., Carenini, G., & Field, T. S. (2020) · Computer Speech & Language · doi:10.1016/j.csl.2020.101181 · Cited by 22
Pan, Y. (2021) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.7127724 · Cited by 18
de la Fuente Garcia, S., Ritchie, C. W., & Luz, S. (2020) · Journal of Alzheimer's Disease · doi:10.3233/jad-200888 · Cited by 5
Comparing Acoustic-based Approaches for Alzheimer's Disease DetectionPreprint
Balagopalan, A., & Novikova, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.01555 · Cited by 1
Research works printed but not linked: 9
- H Goodglass, 2001, BDAE: The Boston Diagnostic Aphasia Examination.
- S Luz, 2020, Alzheimer’s Dementia Recognition through Spontaneous Speech: The ADReSS Challenge [Internet]., arXiv
- SO Orimaye, 2015, Learning Linguistic Biomarkers for Predicting Mild Cognitive Impairment using Compound Skip-grams [Internet]., arXiv
- Y Aytar, 2016, SoundNet: Learning Sound Representations from Unlabeled Video. In: Advances in Neural Information Processing Systems [Internet].
- A Balagopalan, 2020, To BERT or Not To BERT: Comparing Speech and Language-based Approaches for Alzheimer’s Disease Detection [Internet]., arXiv
- J Devlin, 2019, BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding [Internet]., arXiv
- A Radford, 2018, Improving language understanding by generative pre-training
- T Brown, 2020, Language Models are Few-Shot Learners. In: Advances in Neural Information Processing Systems [Internet]., 1877
- ME Peters, 2018, Deep contextualized word representations [Internet]., arXiv
Cited by
Large language models in healthcare and medical domain: A review
Nazi, Z. A., & Peng, W. (2024) · Informatics · doi:10.3390/informatics11030057 · Cited by 415
Haltaufderheide, J., & Ranisch, R. (2024) · npj Digital Medicine · doi:10.1038/s41746-024-01157-x · Cited by 396
“38–”
Hirosawa, T., Harada, Y., Yokose, M., Sakamoto, T., Kawamura, R., & Shimizu, T. (2023) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph20043378 · Cited by 343
“Some GPT-3-related medical articles have been reported about a medical dialogue summarizer [28], navigation application for electronic health record systems [5], and its implementation in clinical use [7] including ophthalmology [4] and dementia prediction [29].”
BackgroundMoulaei, K., Yadegari, A., Baharestani, M., Farzanbakhsh, S., Sabet, B., & Reza Afrash, M. (2024) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2024.105474 · Cited by 244
Wang, L., Wan, Z., Ni, C., Song, Q., Li, Y., Clayton, E., Malin, B., & Yin, Z. (2024) · Journal of Medical Internet Research · doi:10.2196/22769 · Cited by 165
“ChatGPT has been evaluated in dental [6], allergy [64], and mental disorders diagnoses [66].”
Metadata
- Authors
- Agbavor, F., & Liang, H.
- Affiliations
- Drexel University (US)
- Corresponding author
- Liang, Hualou
- Venue
- PLOS Digital Health
- Publisher
- Public Library of Science (PLoS)
- Published
- 2022-12-22
- Volume
- 1
- Issue
- 12
- Pages
- e0000168
- ISSN
- 2767-3170 (ISSN-L)
- PMC
- PMC9931366
- OpenAlex
- W4312091558
- Cited by
- 147
- References
- 34
- Citations by year
- 2026: 39 · 2025: 47 · 2024: 42 · 2023: 17 · 2022: 2
- FWCI
- 17.60
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- PLOS Digital Health (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Funders
- Drexel University
- National Institutes of Health (AG05133)
- National Institute on Aging (P50AG005133)
- Topics
- Health Sciences › Medicine › Psychiatry and Mental health › Dementia and Cognitive Impairment Research
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Keywords
- dementia prediction, Alzheimer's disease, spontaneous speech, GPT-3, text embeddings, language impairment, early diagnosis, speech biomarkers, natural language processing
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.