Listener agent for elderly people with dementia
The DOI matches its Crossref record; 6 of the 7 references it cites were found in a registry or an index.
Abstract
With the goal of developing a conversational humanoid that can serve as a companion for people with dementia, we propose an autonomous virtual agent that can generate backchannel feedback, such as head nods and verbal acknowledgement, on the basis of acoustic information in the user's speech. The system is also capable of speech recognition and language understanding functionalities, which are potentially useful for evaluating the cognitive status of elderly people on a daily basis.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“For example, Sakai et al. found that participants were more interested in interacting with artificial conversational agents that produced social signals, such as smiles and nods [35].”
From A Conversational Robot for Older Adults with Alzheimer's Disease (2020) · cites this paper as [35]
In the citing paper
“Lastly, the authors we cited above showed that people with MCI or dementia successfully interacted with ECAs, whether through remote control button presses or social-like interaction.”
From Field evaluation with cognitively-impaired older adults of attention management in the Embodied Conversational Agent Louise (2016) · cites this paper as [11]
Citations
Cites
Embodied Conversational Agents
(2000) · The MIT Press eBooks · doi:10.7551/mitpress/2697.001.0001 · Cited by 1,799
Gratch, J., Okhmatovskaia, A., Lamothe, F., Marsella, S., Morales, M., van der Werf, R. J., & Morency, L.-P. (2006) · Lecture Notes in Computer Science · doi:10.1007/11821830_2 · Cited by 247
Nudge Nudge Wink Wink: Elements of Face-to-Face Conversation for Embodied Conversational Agents
Cassell, J. (2000) · Embodied Conversational Agents · doi:10.7551/mitpress/2697.003.0002 · Cited by 233
Bickmore, T. W., & Picard, R. W. (2004) · CHI '04 Extended Abstracts on Human Factors in Computing Systems · doi:10.1145/985921.986097 · Cited by 122 · cited 1×
“Besides providing physical assistance to those with physical impairments, it is important for assistive robots to provide communication functions [1].”
(2006) · Cited by 93
Responding to subtle, fleeting changes in the user's internal state
Tsukahara, W., & Ward, N. (2001) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/365024.365047 · Cited by 47 · cited 1×
“The feedback generation rules were created on the basis of previous studies [5, 6].”
Method
Research works printed but not linked: 2
- Pollack M., 2005, Magazine., 9
- Heerink M., 2006, International Journal of ARM, 7
Cited by
Whelan, S., Murphy, K., Barrett, E., Krusche, C., Santorelli, A., & Casey, D. (2018) · International Journal of Social Robotics · doi:10.1007/s12369-018-0471-x · Cited by 162
“Sakai et al (2012) [58] describe an autonomous virtual agent, capable of speech recognition, which can nod its head, providing verbal acknowledgment to users.”
Detecting Dementia Through Interactive Computer Avatars
Tanaka, H., Adachi, H., Ukita, N., Ikeda, M., Kazui, H., Kudo, T., & Nakamura, S. (2017) · IEEE Journal of Translational Engineering in Health and Medicine · doi:10.1109/jtehm.2017.2752152 · Cited by 127
“A different computer avatar system was capable of speech-recognition and language-understanding functionalities, which are potentially useful for evaluating the daily cognitive status of seniors [17].”
BackgroundVirtual Agents as Daily Assistants for Elderly or Cognitively Impaired People
Yaghoubzadeh, R., Kramer, M., Pitsch, K., & Kopp, S. (2013) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-40415-3_7 · Cited by 98
“[11] evaluated a Flash-based virtual agent providing back-channel feedback to monologues produced by elderly patients suffering from dementia, whom they reported as being “willing to be engaged in conversation” with the system.”
BackgroundThe Value of Experience-Centred Design Approaches in Dementia Research Contexts
Morrissey, K., McCarthy, J., & Pantidi, N. (2017) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3025527 · Cited by 95
“Although most of these papers proceed from a progressive, psychosocial perspective on dementia, many also lack actual connection and presence between two or more people – instead, several of these systems intend to automate processes such as choosing material for reminiscence sessions or even conversations (such as Sa…”
BackgroundConversational Affective Social Robots for Ageing and Dementia Support
Lima, M. R., Wairagkar, M., Gupta, M., Rodriguez y Baena, F., Barnaghi, P., Sharp, D. J., & Vaidyanathan, R. (2022) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2021.3115228 · Cited by 75
“Previous research has explored the use of conversational agents with individuals with MCI and dementia; pause and utterance duration, pitch, frequency of head nods, and overall patient responsiveness in the conversation were used as indicators of cognitive status [118]–[120].”
Irfan, B., Kuoppamäki, S., & Skantze, G. (2024) · Frontiers in Robotics and AI · doi:10.3389/frobt.2024.1363713 · Cited by 97
“In addition, generating social signals that match the robot’s utterances can improve the believability, perceived friendliness, and politeness of the robot, and increase user interest in interacting with the robot (Sakai et al., 2012; Fischer et al., 2019).”
BackgroundA Conversational Robot for Older Adults with Alzheimer's Disease
Pou-Prom, C., Raimondo, S., & Rudzicz, F. (2020) · ACM Transactions on Human-Robot Interaction · doi:10.1145/3380785 · Cited by 59
“found that participants were more interested in interacting with artificial conversational agents that produced social signals, such as smiles and nods [35].”
BackgroundMultimodal Wearable Intelligence for Dementia Care in Healthcare 4.0: a Survey
Yang, P., Bi, G., Qi, J., Wang, X., Yang, Y., & Xu, L. (2021) · Information Systems Frontiers · doi:10.1007/s10796-021-10163-3 · Cited by 62
“…technologies that can capture user context, i.e. a user's situation, with all relevant aspects, including activity and behaviour , emotion (Sano & Picard, 2013), location (Seon-Woo Lee & Mase, 2002), and social interaction (Paradiso et al., 2010), as well as provide interaction continuously and…”
Ray, P. P., Dash, D., & De, D. (2019) · Journal of Medical Systems · doi:10.1007/s10916-019-1417-z · Cited by 57
Dialogue Systems and Conversational Agents for Patients with Dementia: The Human–Robot Interaction
Russo, A., D'Onofrio, G., Gangemi, A., Giuliani, F., Mongiovi, M., Ricciardi, F., Greco, F., Cavallo, F., Dario, P., Sancarlo, D., et al. (2019) · Rejuvenation Research · doi:10.1089/rej.2018.2075 · Cited by 47
“Nonaka et al. 53 Huang et al. 54 Sakai et al. 55 VCA-SA Pause duration, utterance duration, pitch, and frequency of head nods related to patient’s responsiveness in the conversation, as indicators of patient’s cognitive status.”
A Virtual Assistive Companion for Older Adults: Design Implications for a Real-World Application
Tsiourti, C., Ben Moussa, M., Quintas, J., Loke, B., Jochem, I., Lopes, J. A., & Konstantas, D. (2018) · Lecture Notes in Networks and Systems · doi:10.1007/978-3-319-56994-9_69 · Cited by 45
Wargnier, P., Carletti, G., Laurent-Corniquet, Y., Benveniste, S., Jouvelot, P., & Rigaud, A.-S. (2016) · 2016 IEEE International Conference on Serious Games and Applications for Health (SeGAH) · doi:10.1109/segah.2016.7586282 · Cited by 28
The CaMeLi Framework—A Multimodal Virtual Companion for Older Adults
Tsiourti, C., Quintas, J., Ben-Moussa, M., Hanke, S., Nijdam, N. A., & Konstantas, D. (2018) · Studies in Computational Intelligence · doi:10.1007/978-3-319-69266-1_10 · Cited by 25
Making Turn-Taking Decisions for an Active Listening Robot for Memory Training
Johansson, M., Hori, T., Skantze, G., Höthker, A., & Gustafson, J. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-47437-3_92 · Cited by 15
“[3] proposed an ECA as a conversational partner for individuals with dementia, using a combination of questions asked by the ECA and a rule-based system to generate verbal and non-verbal backchannel feedback, based on silence length and pitch.”
BackgroundMethodPower in Human-Robot Interaction
Hou, Y. T.-Y., Cheon, E., & Jung, M. F. (2024) · ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1145/3610977.3634949 · Cited by 53
Dixon, E., Michaels, R. B., Xiao, X. H., Zhong, Y., Clary, P., Narayanan, A., Brewer, R. N., & Lazar, A. (2022) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/3517428.3544809 · Cited by 37
“This provides a diferent perspective from past work in AI and dementia, which has primarily focused on ways AI could be used to detect and monitor the progress of dementia [8, 41, 87, 92, 96], in smart home environments to support care partners in monitoring the activities of individuals with dementia [3, 24, 26, 63,…”
BackgroundMakasi, T., Nili, A., Desouza, K. C., & Tate, M. (2022) · Hawaii International Conference on System Sciences (HICSS) · doi:10.24251/hicss.2022.292 · Cited by 29
Negative affect in human robot interaction — Impoliteness in unexpected encounters with robots
Rehm, M., & Krogsager, A. (2013) · IEEE RO-MAN · doi:10.1109/roman.2013.6628529 · Cited by 21
“Following the paradigm of listener agents to establish a relationship with users, several studies on virtual agents have shown that this paradigm has a good potential of building rapport and engaging the user in prolonged interactions ([1]; [2]; [3]).”
BackgroundA framework for the assessment of synthetic personalities according to user perception
Callejas, Z., Griol, D., & López-Cózar, R. (2014) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2014.02.002 · Cited by 25
Krogsager, A., Segato, N., & Rehm, M. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-07230-2_62 · Cited by 21
Wargnier, P., Malaise, A., Jacquemot, J., Benveniste, S., Jouvelot, P., Pino, M., & Rigaud, A.-S. (2015) · 2015 3rd IEEE VR International Workshop on Virtual and Augmented Assistive Technology (VAAT) · doi:10.1109/vaat.2015.7155406 · Cited by 18
“In [19], Sakai et al. present a listener ECA that includes speech input processing and non-verbal feedback for people with dementia.”
Lehmann, J., Granrath, L., Browne, R., Ogawa, T., Kokubun, K., Taki, Y., Jokinen, K., Janböcke, S., Lohr, C., Wieching, R., et al. (2024) · Sustainability · doi:10.3390/su16073064 · Cited by 19
(2013) · Cited by 16
Jordan, P., Silva, P. A., Nunes, F., & Oliveira, R. (2013) · Hawaii International Conference on System Sciences (HICSS) · doi:10.1109/hicss.2013.14 · Cited by 13
“cognitive games to decelerate the course of the disease [18], affective virtual agents (computer characters that recognize sentiment and communicate using gestures or voice output) to recognize the mood of the PwD [19], reminiscence therapy with photo and video”
BackgroundAn anime agent system for reminiscence therapy
Yasuda, K., Fuketa, M., & Aoe, J. (2014) · Gerontechnology · doi:10.4017/gt.2014.13.02.239.00 · Cited by 10
Self-citationImplementation and Evaluation of Interactive Memory-Aid Agent Service for People with Dementia
Tokunaga, S., Horiuchi, H., Takatsuka, H., Saiki, S., Matsumoto, S., Nakamura, M., & Yasuda, K. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-40247-5_36 · Cited by 10
“Nowadays, some systems have the potential to cover the subset of tasks which caregivers have, in order to complement the lack of human resources [8].”
BackgroundSelf-citationTalk with Ted: An embodied conversational agent for caregivers
Curumsing, M. K., Rivera‐Villicana, J., Vouliotis, A., Burns, K., Babaei, M., Petrovich, T., Mouzakis, K., & Vasa, R. (2024) · Gerontology & Geriatrics Education · doi:10.1080/02701960.2024.2302584 · Cited by 7
“In the context of dementia, Sakai et al. (Sakai, Nonaka, Yasuda, & Nakano, 2012) developed an ECA to serve as a companion for people with dementia and support them with verbal feedback and head nods.”
Selecting Popular Topics for Elderly People in Conversation-based Companion Agents
Tsukada, K., Takase, Y., & Nakano, Y. (2015) · ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1145/2701973.2702015 · Cited by 6
“When speech signal is detected, this module determines the backchannel behaviors based on the pitch information as implemented in [4].”
BackgroundMethodSelf-citationAssessing the communication attitude of the elderly using prosodic information and head motions
Yamanaka, T., Takase, Y., & Nakano, Y. (2016) · ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1109/hri.2016.7451845 · Cited by 4
“1 shows an interaction between an older adult and our companion agent [4].”
BackgroundSelf-citationTechnology-based solutions to address the family care gap challenge
Demiris, G. (2021) · Elsevier eBooks · doi:10.1016/b978-0-12-813898-4.00012-9 · Cited by 4
Using 3D Virtual Agents to Improve the Autonomy and Quality of Life of Elderly People
Garrido, P., Sanchez, A., Martínez, F. J., Baldassarri, S., Cerezo, E., & Arbeloa, F. J. S. (2013) · Advances in intelligent systems and computing · doi:10.1007/978-3-319-00566-9_17 · Cited by 3
“Although the market offers other initiatives based on the us e of ICTs to facilitate access to health services through Internet, Digital Terres trial Television (DTT), or cellphones, the use of Embodied Conversational Agents (ECA s) in this area is not widespread, and it is often centered on domotics [6], or aime d at…”
Artificial Intelligence and Automation
Koenig, S., Chan, S.-H., Li, J., & Zheng, Y. (2023) · Springer handbooks · doi:10.1007/978-3-030-96729-1_9 · Cited by 5
Effects of Conversational Contexts and Forms of Non-lexical Backchannel on User Perception of Robots
Kim, S., Seok, S., Choi, J., Lim, Y., & Kwak, S. S. (2021) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros51168.2021.9636589 · Cited by 5
Controlling the Listener Response Rate of Virtual Agents
de Kok, I., & Heylen, D. (2013) · Lecture notes in computer science · doi:10.1007/978-3-642-40415-3_15 · Cited by 3
Assessing the Communication Attitude of the Elderly using Prosodic Information and Head Motions
Yamanaka, T., Takase, Y., & Nakano, Y. (2016) · doi:10.5555/2906831.2906966 · Cited by 2
Self-citationBackchannel Prediction for Mandarin Human-Computer Interaction
Mao, X., Peng, Y., Xue, Y., Na, L. U. O., & Rovetta, A. (2015) · IEICE Transactions on Information and Systems · doi:10.1587/transinf.2014edp7214 · Cited by 2
“Sakai et al. developed a listener agent for elderly people with dementia [15].”
BackgroundLaughter as the Best Medicine: Exploring Humour-Mediated Health Applications
Dormann, C. (2015) · Interacción · doi:10.1007/978-3-319-20804-6_58 · Cited by 2
“Studies on the acceptance of robots and agents by seniors show that an agent who displays social signals is better able to gain users’ trust and interest [2].”
BackgroundSegato, N., Krogsager, A., Jensen, D. G., & Rehm, M. (2014) · Communications in computer and information science · doi:10.1007/978-3-319-07854-0_101 · Cited by 1
Videophone Conversation of Two Individuals with Dementia Using an Anime Agent System
Yasuda, K., Fuketa, M., Morita, K., Aoe, J.-I., & Kuwahara, N. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-40247-5_32 · Cited by 1
Self-citationIntelligent Support Technologies for Older People: An Analysis of Characteristics and Roles
Petrie, H., Darzentas, J. S., & Carmien, S. P. (2018) · Springer eBooks · doi:10.1007/978-3-319-75028-6_8 · Cited by 2
“2016) to indicate caring and communicative goals and listener agent (Sakai et al. 2012) to indicate an ECA which can detect the cognitive status of older people with dementia.”
BackgroundImproving face recognition of artificial social companions for smart working and living environments
Quintas, J. M. (2019) · International Conference on PErvasive Technologies Related to Assistive Environments (PETRA) · doi:10.1145/3316782.3324014 · Cited by 1
“A variety of ASCs has been developed aiming to provide social support to isolated older adults [19, 20] and to address daily needs for an autonomous living.”
BackgroundCognitive computing systems and public value: The case of chatbots and public service delivery
Makasi, T. (2022) · Queensland University of Technology · doi:10.5204/thesis.eprints.230002 · Cited by 2
Involvement of caregiver in decision-making to administer tube feeding to elderly person
Mizuoka, T., & Fujinami, T. (2014) · Gerontechnology · doi:10.4017/gt.2014.13.02.237.00
Avatars 4 All – An Avatar Generation Toolchain
Sili, M., Broneder, E., Morandell, M. M., & Mayer, C. C. (2016) · International Conference on Pervasive Computing Technologies for Healthcare · doi:10.4108/eai.16-5-2016.2264655 · Cited by 1
Building Table-Talk Agents that Create a Pl easant Atmosphere
Yuasa, M. (2013)
Information-gathering dialog system using acoustic features and user’s motivation
栂井, 良., Tsunakawa, T., Nishida, M., & Nishimura, M. (2022) · IAES International Journal of Artificial Intelligence · doi:10.11591/ijai.v12.i1.pp12-22
“To solve this problem, a chat dialog system has attracted widespread attention as a method for keeping their purpose in life and watching their daily lives [3], [4].”
Towards Self Expression Through Voice User Interfaces
Stigall, B. (2022) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/1071181322661048 · Cited by 1
LLM Enabled Social Robots – Transforming Aged Care Through AI
Tahir, H., Ansari, S., Bushra, N., Khan, A., Lwin, M., Alnajjar, F., & Mubin, O. (2025) · International Conference on Activity and Behavior Computing (ABC) · doi:10.1109/abc64332.2025.11118445
“One such example is the work from, Sakai et al. where the study depicts that the participants were more engaged when robots exhibited social signals such as smiles and nods compared to non-emotion based interation [10].”
Metadata
- Authors
- Sakai, Y., Nonaka, Y., Yasuda, K., & Nakano, Y. I.
- Affiliations
- Seikei University (JP)
- Chiba Rosai Hospital (JP)
- ORCID
- Nakano, Yukiko I.
- Venue
- Proceedings of the seventh annual ACM/IEEE international conference on Human-Robot Interaction
- Publisher
- ACM
- Published
- 2012-03-05
- Pages
- 199-200
- OpenAlex
- W1993901015
- Cited by
- 47
- References
- 7
- Citations by year
- 2025: 1 · 2024: 4 · 2023: 1 · 2022: 5 · 2021: 4 · 2019: 2 · 2018: 3 · 2017: 4 · 2016: 7 · 2015: 4 · 2014: 5 · 2013: 7
- FWCI
- 3.93
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM/IEEE International Conference on Human-Robot Interaction (HRI) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
- Physical Sciences › Computer Science › Artificial Intelligence › Emotion and Mood Recognition
- Keywords
- dementia, older adults, companion robots, head nodding, speech recognition, language comprehension, cognitive status assessment
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.