Identifying and Avoiding Confusion in Dialogue with People with Alzheimer's Disease
Open accessThe DOI matches its Crossref record; 65 of the 66 references it cites were found in a registry or an index.
Abstract
Alzheimer's disease (AD) is an increasingly prevalent cognitive disorder in which memory, language, and executive function deteriorate, usually in that order. There is a growing need to support individuals with AD and other forms of dementia in their daily lives, and our goal is to do so through speech-based interaction. Given that 33% of conversations with people with middle-stage AD involve a breakdown in communication, it is vital that automated dialogue systems be able to identify those breakdowns and, if possible, avoid them. In this article, we discuss several linguistic features that are verbal indicators of confusion in AD (including vocabulary richness, parse tree structures, and acoustic cues) and apply several machine learning algorithms to identify dialogue-relevant confusion from speech with up to 82% accuracy. We also learn dialogue strategies to avoid confusion in the first place, which is accomplished using a partially observable Markov decision process and which obtains accuracies (up to 96.1%) that are significantly higher than several baselines. This work represents a major step towards automated dialogue systems for individuals with dementia.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures













Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In [108], ML algorithms have been applied to identify dialog-related confusion from speech with individuals with Alzheimer's disease; accuracies above 80% were obtained and learn policies implemented to avoid conversation breakdowns.”
From Conversational Affective Social Robots for Ageing and Dementia Support (2022) · cites this paper as [108]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Some recent work has used reinforcement learning to learn policies that can at least limit those instances [5].”
From A Conversational Robot for Older Adults with Alzheimer's Disease (2020) · cites this paper as [5]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Several linguistic features were extracted as verbal indicators of confusion (e.g., vocabulary richness, parse tree structures, and acoustic cues).”
From Conversational Affective Social Robots for Ageing and Dementia Support (2022) · cites this paper as [108]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Folstein, M. F., Folstein, S. E., & McHugh, P. R. (1975) · Journal of Psychiatric Research · doi:10.1016/0022-3956(75)90026-6 · Cited by 88,282 · cited 1×
“…provides a unified and validated scale for measuring the severity of dementia on a scale of 0 (greatest cognitive decline) to 30 (no cognitive decline), based on a series of questions in five areas: orientation, registration, attention, memory, and language (Folstein, Folstein, and McHugh 1975).”
BackgroundSMOTE synthetic minority over-sampling technique
Chawla, N. V., Bowyer, K. W., Hall, L. O., & Kegelmeyer, W. P. (2002) · Journal of Artificial Intelligence Research · doi:10.1613/jair.953 · Cited by 32,865 · cited 1×
“In this experiment, we generate synthetic data points using the synthetic minority oversampling technique (SMOTE), which under-samples the majority class and over-samples the minority class by creating synthetic minority class examples (Chawla et al. 2002).”
MethodWordNet: a lexical database for English
Miller, G. A. (1995) · Communications of the ACM · doi:10.1145/219717.219748 · Cited by 14,015
The Preference for Self-Correction in the Organization of Repair in Conversation
Schegloff, E. A., Jefferson, G., & Sacks, H. (1977) · Language · doi:10.2307/413107 · Cited by 5,150 · cited 1×
“The trouble source-repair (TSR) model describes difficulties in communication in any dyads, including how repairs are initiated and carried out (Schegloff, Jefferson, and Sacks 1977).”
MethodRecognizing contextual polarity in phrase-level sentiment analysis
Wilson, T., Wiebe, J., & Hoffmann, P. (2005) · Proceedings of the conference on Human Language Technology and Empirical Methods in Natural Language Processing - HLT '05 · doi:10.3115/1220575.1220619 · Cited by 3,345 · cited 1×
“We also compute objectivity measures using the MPQA Subjectivity Lexicon (Wilson, Wiebe, and Hoffmann 2005), which categorizes words by their subjectivity (strong or weak) and their prior, out-of-context, polarity (positive, negative, both, or neutral).”
MethodThe assessment of aphasia and related disorders
Goodglass, H., & Kaplan, E. (1972) · Medical Entomology and Zoology · Cited by 3,258
Accurate unlexicalized parsing
Klein, D., & Manning, C. D. (2003) · Proceedings of the 41st Annual Meeting on Association for Computational Linguistics - ACL '03 · doi:10.3115/1075096.1075150 · Cited by 3,008 · cited 1×
“Lu’s Syntactic Complexity Analyzer was initially designed to measure the syntactic complexity of second-language writing and, in turn, uses the Stanford parser (Klein and Manning 2003).”
MethodFeature-rich part-of-speech tagging with a cyclic dependency network
Toutanova, K., Klein, D., Manning, C. D., & Singer, Y. (2003) · Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology - NAACL '03 · doi:10.3115/1073445.1073478 · Cited by 2,844 · cited 2×
“We include various part-of-speech (POS) features, such as the proportion of nouns, verbs, light verbs, adjectives, adverbs, prepositional words, demonstratives, functional words generally, and the noun-to-verb ratio, obtained using the Stanford POS tagger (Toutanova et al. 2003).”
MethodBrysbaert, M., & New, B. (2009) · Behavior Research Methods · doi:10.3758/brm.41.4.977 · Cited by 2,821
SWITCHBOARD: telephone speech corpus for research and development
Godfrey, J. J., Holliman, E. C., & McDaniel, J. (1992) · [Proceedings] ICASSP-92: 1992 IEEE International Conference on Acoustics, Speech, and Signal Processing · doi:10.1109/icassp.1992.225858 · Cited by 2,134 · cited 1×
“To estimate statistical deviations from typical language, we also compute the average log probability of each word in each utterance given two trigram models with Good-Turing smoothing, from the Brown and Switchboard (Godfrey, Holliman, and McDaniel 1992) corpora, respectively.”
Method
Research works printed but not linked: 2
- 16. Choi, J. and K.-E. Kim. 2011. Inverse reinforcement learning in partially observable environments. Journal of Machine Learning Research, 12:691-730.
- 33. Kempler, D. 1995. Language changes in dementia of the Alzheimer type. In R. Lubinski, editor, Dementia and Communication: Research and Clinical Implications, pages 98-114, Singular, San Diego, CA.
Cited by
Artificial intelligence and the implementation challenge
Shaw, J., Rudzicz, F., Jamieson, T., & Goldfarb, A. (2019) · Journal of Medical Internet Research · doi:10.2196/13659 · Cited by 389
“…ML can dynamically adapt survey questions to more quickly identify possible diseases [17], dynamically avoid potential communication breakdowns during speech conversation in the assessment of dementia [18], and even recommend treatments directly when using structured institutional data [19].”
BackgroundSelf-citationLinguistic features and automatic classifiers for identifying mild cognitive impairment and dementia
Calzà, L., Gagliardi, G., Rossini Favretti, R., & Tamburini, F. (2021) · Computer Speech & Language · doi:10.1016/j.csl.2020.101113 · Cited by 89
Conversational Affective Social Robots for Ageing and Dementia Support
Lima, M. R., Wairagkar, M., Gupta, M., Rodriguez y Baena, F., Barnaghi, P., Sharp, D. J., & Vaidyanathan, R. (2022) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2021.3115228 · Cited by 75
“In [108], ML algorithms have been applied to identify dialog-related confusion from speech with individuals with Alzheimer’s disease; accuracies above 80% were obtained and learn policies implemented to avoid conversation breakdowns.”
BackgroundA Conversational Robot for Older Adults with Alzheimer's Disease
Pou-Prom, C., Raimondo, S., & Rudzicz, F. (2020) · ACM Transactions on Human-Robot Interaction · doi:10.1145/3380785 · Cited by 59
“Some recent work has used reinforcement learning to learn policies that can at least limit those instances [5].”
BackgroundSelf-citationKobayashi, M., Kosugi, A., Takagi, H., Nemoto, M., Nemoto, K., Arai, T., & Yamada, Y. (2019) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-29390-1_4 · Cited by 45
“Rudzicz et al. [39] and Chinaei et al. [40] built a computational model that aimed to exploit linguistic and acoustic features to detect a breakdown in conversations.”
Integrated maintenance and operations decision making with imperfect degradation state observations
Çelen, M., & Djurdjanović, D. (2020) · Journal of Manufacturing Systems · doi:10.1016/j.jmsy.2020.03.010 · Cited by 36
An Automated Approach to Examining Pausing in the Speech of People With Dementia
Sluis, R. A., Angus, D. J., Wiles, J., Back, A. D., Gibson, T., Liddle, J., Worthy, P., Copland, D. A., & Angwin, A. J. (2020) · American Journal of Alzheimer s Disease & Other Dementias® · doi:10.1177/1533317520939773 · Cited by 50
Yamada, Y., Shinkawa, K., & Shimmei, K. (2020) · JMIR Mental Health · doi:10.2196/16790 · Cited by 34
“This reduction is measured by the decrease in adjectives and indicators related to vocabulary richness [11,39].”
BackgroundAI-Atroshi, C., Beulah, J. R., Singamaneni, K. K., Cyril, C. P. D., Neelakandan, S., & Velmurugan, S. (2022) · International Journal of Healthcare Management · doi:10.1080/20479700.2022.2097764 · Cited by 30
RetractedMorales-de-Jesús, V. M., Gómez-Adorno, H., Somodevilla-Garcia, M., & Vilariño, D. (2021) · Healthcare · doi:10.3390/healthcare9081036 · Cited by 31
Bynum, J., Dorr, D. A., Lima, J. C., McCarthy, E. P., McCreedy, E. M., Platt, R. G., & Vydiswaran, V. G. V. (2020) · Journal of the American Geriatrics Society · doi:10.1111/jgs.16617 · Cited by 27
“In addition, a large database of multimedia interactions and transcripts, DementiaBank,(28) is available for the study of communication in dementia patients, and it was used to study NLP techniques to classify and analyze the linguistic characteristics of AD patients.(29-31) Despite these exciting developments in the…”
MethodAugmented Reality for Early Alzheimer's Disease Diagnosis
Vovk, A., Patel, A. X., & Chan, D. (2019) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3290607.3313007 · Cited by 20
“Alzheimer’s disease is characterised by gradual memory loss caused by a preferential loss of brain tissue in the hippocampus, the part of the brain responsible for encoding spatial memories [4] (see Figure 1).”
BackgroundA longitudinal multi-modal dataset for dementia monitoring and diagnosis
Gkoumas, D., Wang, B., Tsakalidis, A., Wolters, M. K., Purver, M., Zubiaga, A., & Liakata, M. (2024) · Language Resources and Evaluation · doi:10.1007/s10579-023-09718-4 · Cited by 25
“the Carolinas dataset contains less directed conversations between people with dementia and health practitioners, and has been used to build models for automatic detection of confusion [21].”
MethodWhelan, B., Angus, D. J., Wiles, J., Chenery, H. J., Conway, E., Copland, D. A., Atay, C., & Angwin, A. J. (2018) · Innovation in Aging · doi:10.1093/geroni/igy034 · Cited by 15
“The automatic analysis of trouble-indicating behaviors (Chinaei et al., 2017; Rudzicz, Chan Currie, Danks, Mehta, & Zhao, 2014) and communication strategies, which enhance conversational engagement in dementia (Atay et al.”
MethodA Systematic Review of NLP for Dementia: Tasks, Datasets, and Opportunities
Peled-Cohen, L., & Reichart, R. (2025) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl.a.35 · Cited by 20
“Studies such as Green et al. (2012) and Chinaei et al. (2017) have developed systems to detect confusion and disfluent speech patterns within transcribed interviews, suggesting real-time repair strategies to minimize misunderstandings.”
Hall, A., Shinkawa, K., Kosugi, A., Takase, T., Kobayashi, M., Nishimura, M., Nemoto, M., Watanabe, R., Tsukada, E., Ota, M., et al. (2019) · PubMed Central · Cited by 17
Alzheimer's Dementia Detection through Spontaneous Dialogue with Proactive Robotic Listeners
Li, Y., Lai, C., Lala, D., Inoue, K., & Kawahara, T. (2022) · ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1109/hri53351.2022.9889375 · Cited by 18
“Their utterances contain many disfluencies, such as fillers, false starts, repetitions, and so on [15], [16].”
BackgroundConversational trouble and repair in dementia: Revision of an existing coding framework
Sluis, R. A., Campbell, A., Atay, C., Conway, E., Mok, Z., Angwin, A. J., Chenery, H. J., & Whelan, B. (2019) · Journal of Communication Disorders · doi:10.1016/j.jcomdis.2019.105912 · Cited by 10
Linguistic Markers in Spontaneous Speech: Insights into Subjective Cognitive Decline (Review)
Segkouli, S., Γκιόκα, Μ., Kokkas, S., Votis, K., Valero, S., Miguel, A., Antoniades, A., Charalambous, E., & Manias, G. (2025) · Healthcare · doi:10.3390/healthcare13222888 · Cited by 11
“These computational methods have already proven successful in detecting the subtle linguistic markers that signal the very early stages of dementia [44–47] and characterizing associated conditions like AD [25,48–50].”
RetroMind and the Image of Memories: A Preliminary Study of a Support Tool for Reminiscence Therapy
Tucci, C., Amaro, I., Greca, A. D., & Tortora, G. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-60615-1_31 · Cited by 5
Transformer-Based Deep Learning Approaches for Speech-Based Dementia Detection: A Systematic Review
Mobtahej, P., Gouron, S., Javaheri, R., Le, D. N., El‐Khoury, A., Naing, T. H., & Sajjadi, S. A. (2025) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2025.3595781 · Cited by 9
Towards Domain-Agnostic and Domain-Adaptive Dementia Detection from Spoken Language
Farzana, S., & Parde, N. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.668 · Cited by 13
“Control and Figure 1: Characteristic language samples from DB (Chinaei et al., 2017) and CCC (Davis et al., 2017). dementia subjects are matched in terms of age and gender, resulting in a balanced set of 156 samples (78 with dementia and 78 controls ) split into training and test.”
BackgroundMethodHuman behavior understanding from motion and bodily cues using deep neural networks
Dotti, D. (2021) · Research Publications (Maastricht University) · doi:10.26481/dis.20210615dd · Cited by 13
Sinclair, J. (2020) · TSpace (University of Toronto) · Cited by 7
Towards a Natural Language-based Dialog Model as an Assistant Tool for Dementia Therapy
Somodevilla, M. J., & Morales-de-Jesús, V. M. (2019) · Research in Computing Science · doi:10.13053/rcs-148-3-16 · Cited by 1
“Chinaei et al. [2] analyzed several linguistics features that are verbal indicators of confusion in AD like vocabulary richness, parse tree structures, and acoustic cues.”
BackgroundOn the Use of Linguistic Features for the Evaluation of Generative Dialogue SystemsPreprint
Berlot-Attwell, I., & Rudzicz, F. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.06335 · Cited by 1
“Now that we have demonstrated that this approach is sound in principle, future work will include the determination of a full set of linguistic features appropriate for dialogue evaluation (particularly features regarding dialogue acts and dialogue breakdowns (Higashinaka et al., 2015; Chinaei et al., 2017)), to extend…”
BackgroundSelf-citationMode Shifts in Conversations with People with Dementia can Cause Confusion
Maclagan, M. (2019) · Gerontology & Geriatrics Studies · doi:10.31031/ggs.2019.04.000600
“[2] :376 found that “33% of conversations with people with middle-stage AD [Alzheimer’s disease] involve a breakdown in communication.”
BackgroundA Conversational Model for the Reminiscence Therapy of Patients with Early Stage of Alzheimer.
Morales-de-Jesús, V. M., & García, M. J. S. (2020) · Cited by 2
Bridging the Gap: Advancements in Interactive Technology to Support Dementia Care – A Scoping Review
Ma, Y., Nordberg, O. E., Zhang, Y., Rongve, A., Bachinski, M., & Fjeld, M. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-30041-6_11
Giri, N., Din, F. U., Shetty, N. K., Hilton, T., Stephan, B. C. M., & Tully, P. J. (2026) · Elsevier eBooks · doi:10.1016/b978-0-443-45156-0.00011-x
Metadata
- Authors
- Chinaei, H., Currie, L. C., Danks, A., Lin, H., Mehta, T., & Rudzicz, F.
- Affiliations
- University of Toronto (CA)
- Venue
- Computational Linguistics
- Publisher
- MIT Press
- Published
- 2017-06
- Volume
- 43
- Issue
- 2
- Pages
- 377-406
- ISSN
- 0891-2017 (ISSN-L)
- 1530-9312
- OpenAlex
- W2602799059
- Semantic Scholar
- 247fff8f5d3281a20f11d28df3c592bc4fa1e553
- Cited by
- 30
- References
- 66
- Citations by year
- 2026: 2 · 2025: 3 · 2024: 2 · 2023: 1 · 2022: 2 · 2021: 4 · 2020: 8 · 2019: 7 · 2018: 1
- FWCI
- 2.61
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- cc-by-nc-nd
- Locations
- Computational Linguistics (publishedVersion, cc-by-nc-nd)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Funders
- University of Pittsburgh
- Carnegie Mellon University
- AGE-WELL
- Alzheimer Society
- National Institutes of Health (AG005133, AG005133 and AG003705)
- Natural Sciences and Engineering Research Council of Canada (RGPIN 435874)
- National Institute on Deafness and Other Communication Disorders (DC008524, R01DC008524)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Health Sciences › Medicine › Psychiatry and Mental health › Dementia and Cognitive Impairment Research
- Social Sciences › Arts and Humanities › Language and Linguistics › Language, Discourse, Communication Strategies
- Keywords
- Alzheimer's disease, dementia, communication breakdown, confusion detection, speech-based interaction, dialogue strategies, lexical diversity, acoustic cues, partially observable Markov decision processes, conversational support
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.