Artificial Emotional Intelligence in Socially Assistive Robots for Older Adults: A Pilot Study
Open accessThe DOI matches its Crossref record; 22 of the 22 references it cites were found in a registry or an index.
Abstract
This paper presents our recent research on integrating artificial emotional intelligence in a social robot (Ryan) and studies the robot's effectiveness in engaging older adults. Ryan is a socially assistive robot designed to provide companionship for older adults with depression and dementia through conversation. We used two versions of Ryan for our study, empathic and non-empathic. The empathic Ryan utilizes a multimodal emotion recognition algorithm and a multimodal emotion expression system. Using different input modalities for emotion, i.e. facial expression and speech sentiment, the empathic Ryan detects users emotional state and utilizes an affective dialogue manager to generate a response. On the other hand, the non-empathic Ryan lacks facial expression and uses scripted dialogues that do not factor in the users emotional state. We studied these two versions of Ryan with 10 older adults living in a senior care facility. The statistically significant improvement in the users' reported face-scale mood measurement indicates an overall positive effect from the interaction with both the empathic and non-empathic versions of Ryan. However, the number of spoken words measurement and the exit survey analysis suggest that the users perceive the empathic Ryan as more engaging and likable.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The PHQ-9: Validity of a brief depression severity measure
Kroenke, K., Spitzer, R. L., & Williams, J. B. W. (2001) · Journal of General Internal Medicine · doi:10.1046/j.1525-1497.2001.016009606.x · Cited by 44,910
Russell, J. A. (1980) · Journal of Personality and Social Psychology · doi:10.1037/h0077714 · Cited by 16,334 · cited 1×
“There are several models of emotions in the literature [51], where Russell’s [52] and Ekman’s [53] are the most common models used in HRI studies [54], [55].”
MethodRobust Real-Time Face Detection
Viola, P., & Jones, M. J. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000013087.49260.fb · Cited by 14,097
Measuring individual differences in empathy: Evidence for a multidimensional approach.
Davis, M. H. (1983) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.44.1.113 · Cited by 10,272
Picard, R. W. (1997) · MIT Press · doi:10.7551/mitpress/1140.001.0001 · Cited by 7,382
Sheikh (1986) · doi:10.1300/j018v05n01_09 · Cited by 7,236
Empathy: its ultimate and proximate bases
Preston, S. D., & de Waal, F. B. M. (2002) · Behavioral and Brain Sciences · doi:10.1017/s0140525x02000018 · Cited by 4,126 · cited 1×
“Empathy is the capacity to (a) share other people’s emotional state or be affected by it, (b) infer the reasons of said emotional state, and (c) adopt other people’s perspectives [24].”
BackgroundAffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wild
Mollahosseini, A., Hasani, B., & Mahoor, M. H. (2019) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2017.2740923 · Cited by 2,338
Self-citationDapretto, M., Davies, M. S., Pfeifer, J. H., Scott, A. A., Sigman, M., Bookheimer, S. Y., & Iacoboni, M. (2006) · Nature Neuroscience · doi:10.1038/nn1611 · Cited by 1,694
Speech emotion recognition using hidden Markov models
Nwe, T. L., Foo, S. W., & De Silva, L. C. (2003) · Speech Communication · doi:10.1016/s0167-6393(03)00099-2 · Cited by 920
Hill, J., Randolph Ford, W., & Farreras, I. G. (2015) · Computers in Human Behavior · doi:10.1016/j.chb.2015.02.026 · Cited by 856
Tariq, S. H., Tumosa, N., Chibnall, J. T., Perry, M. H., & Morley, J. E. (2006) · The American Journal of Geriatric Psychiatry · doi:10.1097/01.jgp.0000221510.33817.86 · Cited by 724
Hess, U., & Blairy, S. (2001) · International Journal of Psychophysiology · doi:10.1016/s0167-8760(00)00161-6 · Cited by 697 · cited 1×
“Some studies suggest that empathy can be traced back to the mirror neuron system [42], [43]; (c) a predefined emotion scripted by a psychologist.”
BackgroundSocially assistive robotics [Grand Challenges of Robotics]
Tapus, A., Mataric, M., & Scassellati, B. (2007) · IEEE Robotics & Automation Magazine · doi:10.1109/mra.2007.339605 · Cited by 547
Pu, L., Moyle, W., Jones, C., & Todorovic, M. (2019) · The Gerontologist · doi:10.1093/geront/gny046 · Cited by 516 · cited 1×
“The development of affective and empathic robots that have the capability to recognize users’ emotions and interact with themnaturally and effectively is in its infancy and more research needs to be carried out in this field [15].”
BackgroundThe influence of empathy in human-robot relations
Leite, I., Pereira, A., Mascarenhas, S., Martinho, C., Prada, R., & Paiva, A. (2013) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2012.09.005 · Cited by 344 · cited 1×
“Recently, there have been several studies that investigate incorporating empathy in social robots [25], [26], [27], [28].”
BackgroundAffective multimodal human-computer interaction
Pantic, M., Sebe, N., Cohn, J. F., & Huang, T. (2005) · Proceedings of the 13th annual ACM international conference on Multimedia · doi:10.1145/1101149.1101299 · Cited by 234
Building autonomous sensitive artificial listeners
Schroder, M., Bevacqua, E., Cowie, R., Eyben, F., Gunes, H., Heylen, D., ter Maat, M., McKeown, G., Pammi, S., Pantic, M., et al. (2012) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2011.34 · Cited by 185
Vandemeulebroucke, T., de Casterlé, B. D., & Gastmans, C. (2018) · Aging & Mental Health · doi:10.1080/13607863.2017.1286455 · Cited by 160 · cited 1×
“SAR has shown to alleviate this problem by providing companionship to assist older adults through conversation and social interaction [4], [5].”
BackgroundChen, S.-C., Moyle, W., Jones, C., & Petsky, H. (2020) · International Psychogeriatrics · doi:10.1017/s1041610220000459 · Cited by 153 · cited 1×
“The COVID-19 pandemic has highlighted the healthcare worker shortage that currently plagues the healthcare sys-tem [6], and SAR has recently been used to address this problem by researchers [7], [8], [9].”
MethodDesigning emotionally sentient agents
McDuff, D., & Czerwinski, M. (2018) · Communications of the ACM · doi:10.1145/3186591 · Cited by 132
Delivering Cognitive Behavioral Therapy Using A Conversational Social Robot
Dino, F., Zandie, R., Abdollahi, H., Schoeder, S., & Mahoor, M. H. (2019) · 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros40897.2019.8968576 · Cited by 83 · cited 2×
“Dino et al. [33] studied the use of a social robot to deliver Internet-based Cognitive Behavioural Therapy (iCBT) to adults with depression.”
BackgroundSelf-citation
Cited by
Large language models are proficient in solving and creating emotional intelligence tests
Schlegel, K., Sommer, N. R., & Mortillaro, M. (2025) · Communications Psychology · doi:10.1038/s44271-025-00258-x · Cited by 46
Promising directions for human-robot interactions defined by older adults
Ostrowski, A. K., Zhang, J., Breazeal, C., & Park, H. W. (2024) · Frontiers in Robotics and AI · doi:10.3389/frobt.2024.1289414 · Cited by 20
Metadata
- Authors
- Abdollahi, H., Mahoor, M. H., Zandie, R., Siewierski, J., & Qualls, S. H.
- Venue
- IEEE Transactions on Affective Computing
- Published
- 2023-07-01
- Volume
- 14
- Issue
- 3
- Pages
- 2020-2032
- ISSN
- 1949-3045 (ISSN-L)
- 2371-9850
- OpenAlex
- W4210814395
- Cited by
- 196
- References
- 22
- FWCI
- 28.07
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.