Emoty: an Emotionally Sensitive Conversational Agent for People with Neurodevelopmental Disorders
Open accessAbstract
Our research aims at exploiting the advances in conversational technology to support people with Neurodevelopmental Disorder (NDD). NDD is a group of conditions that are characterized by severe deficits in the cognitive, emotional and motor areas and produce severe impairments in communication and social functioning. This paper presents the design, technology and exploratory evaluation of Emoty, a spoken Conversational Agent (CA) created specifically for individuals with NDD. The goal of Emoty is to help these persons enhancing communication abilities related to emotional recognition and expression, which are fundamental in any form of human relationship. The system exploits emotion detection capabilities based on the semantics of the speech by calling the IBM Watson Tone Analyzer API and from the harmonic features of the audio thanks to an “all-of-us” Deep Learning model. The design and evaluation of Emoty are based on the close collaboration among computer engineers and specialists in NDD (psychologists, neurological doctors, educators).
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Chatbots have been suggested as accessible interactive systems for visually impaired users, as conversational support for users with special needs, and to support youth engagement in societal issues.
“Chatbots have been suggested as accessible interactive systems for visually impaired in need of an easily navigable user interface [7], as conversational support for users with special needs [19], and to support youth to engage in societal issues [110].”
From Future directions for chatbot research: An interdisciplinary research agenda (2021) · cites this paper as [19]
Voice-based conversational agents are difficult to use for user groups with speech impairments.
“and also to be difficult to use for user groups with speech impairments [19].”
From Future directions for chatbot research: An interdisciplinary research agenda (2021) · cites this paper as [19]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Diagnostic and Statistical Manual of Mental Disorders
Williams, J. B. W., & First, M. B. (2013) · Oxford University Press eBooks · doi:10.1093/acrefore/9780199975839.013.104 · Cited by 109,841 · see citing works
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126 · see citing works
(1983) · Cited by 33,192 · see citing works
ELIZA—a computer program for the study of natural language communication between man and machine
Weizenbaum, J. (1966) · Communications of the ACM · doi:10.1145/365153.365168 · Cited by 4,452 · see citing works
ELIZA — a computer program for the study of natural language communication between man and machine
Weizenbaum, J. (1983) · Communications of the ACM · doi:10.1145/357980.357991 · Cited by 2,765 · see citing works
Vocal communication of emotion: A review of research paradigms
Scherer, K. (2003) · Speech Communication · doi:10.1016/s0167-6393(02)00084-5 · Cited by 2,044 · see citing works
Plutchik, R. (2001) · American Scientist · doi:10.1511/2001.4.344 · Cited by 1,428 · see citing works
Teaching and Learning with Technology: Effectiveness of ICT Integration in Schools
Ghavifekr, S., & Rosdy, W. A. W. (2015) · International Journal of Research in Education and Science · doi:10.21890/ijres.23596 · Cited by 1,160 · see citing works
Exposure and use of mobile media devices by young children
Kabali, H. K., Irigoyen, M. M., Nunez-Davis, R., Budacki, J. G., Mohanty, S. H., Leister, K. P., & Bonner, R. L. (2015) · Pediatrics · doi:10.1542/peds.2015-2151 · Cited by 1,053 · see citing works
Lumley, M. A., Neely, L. C., & Burger, A. J. (2007) · Journal of Personality Assessment · doi:10.1080/00223890701629698 · Cited by 535 · see citing works
Silver, M., & Oakes, P. (2001) · Autism · doi:10.1177/1362361301005003007 · Cited by 455 · see citing works
Pradhan, A., Mehta, K., & Findlater, L. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174033 · Cited by 448 · cited 1× · see citing works
“Furthermore, a recent exploratory study used Amazon Echo in speech rehabilitation [25] for users with cognitive disorders.”
Research Methods in Human-Computer Interaction
Landauer, T. (1988) · Elsevier eBooks · doi:10.1016/b978-0-444-70536-5.50047-6 · Cited by 384 · see citing works
Beaumont, R., & Sofronoff, K. (2008) · Journal of Child Psychology and Psychiatry · doi:10.1111/j.1469-7610.2008.01920.x · Cited by 384 · see citing works
Emoticons and Online Message Interpretation
Derks, D., Bos, A. E. R., & von Grumbkow, J. (2007) · Social Science Computer Review · doi:10.1177/0894439307311611 · Cited by 337 · see citing works
Tanaka, J. W., Wolf, J. M., Klaiman, C., Koenig, K., Cockburn, J., Herlihy, L., Brown, C., Stahl, S., Kaiser, M. D., & Schultz, R. T. (2010) · Journal of Child Psychology and Psychiatry · doi:10.1111/j.1469-7610.2010.02258.x · Cited by 318 · see citing works
Early intervention in neurodevelopmental disorders: underlying neural mechanisms
Cioni, G., Inguaggiato, E., & Sgandurra, G. (2016) · Developmental Medicine & Child Neurology · doi:10.1111/dmcn.13050 · Cited by 214 · see citing works
Patterns for How Users Overcome Obstacles in Voice User Interfaces
Myers, C., Furqan, A., Nebolsky, J., Caro, K., & Zhu, J. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173580 · Cited by 213 · cited 1× · see citing works
“Some studies have observed that the “invisible” nature of spoken conversational technology might make it difficult to learn and use [38].”
BackgroundExposure and Use of Mobile Media Devices by Young Children
(2015) · PEDIATRICS · doi:10.1542/peds.2015-2151d · Cited by 93 · see citing works
Madsen, M., el Kaliouby, R., Goodwin, M. S., & Picard, R. W. (2008) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/1414471.1414477 · Cited by 89 · see citing works
Bouck, E. C., Savage, M., Meyer, N. K., Taber‐Doughty, T., & Hunley, M. (2014) · Focus on Autism and Other Developmental Disabilities · doi:10.1177/1088357614528797 · Cited by 77 · see citing works
Convey: Exploring the Use of a Context View for Chatbots
Jain, M., Kota, R., Kumar, P., & Patel, S. N. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174042 · Cited by 74 · see citing works
Rachel: Design of an emotionally targeted interactive agent for children with autism
Mower, E., Black, M. P., Flores, E., Williams, M. E., & Narayanan, S. S. (2011) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme.2011.6011990 · Cited by 72 · see citing works
Technology Tools for Students with Autism: Innovations that Enhance Independence and Learning
Alley‐Young, G. (2016) · Canadian Journal of Communication · doi:10.22230/cjc.2016v41n3a3046 · Cited by 37 · see citing works
Schuller, B. W., Marchi, E., Baron‐Cohen, S., O’Reilly, H., Robinson, P., Davies, I., Golan, O., Friedenson, S., Tal, S., Newman, S., et al. (2013) · mediaTUM – the media and publications repository of the Technical University Munich (Technical University Munich) · Cited by 28 · see citing works
Schuller, B. W., Marchi, E., Baron‐Cohen, S., Lassalle, A., O’Reilly, H., Pigat, D., Robinson, P., Davies, I., Baltrušaitis, T., Mahmoud, M., et al. (2015) · ENLIGHTEN (Jurnal Bimbingan dan Konseling Islam) · Cited by 26 · see citing works
Designing voice interaction for people with physical and speech impairments
Derboven, J., Huyghe, J., & Grooff, D. D. (2014) · Nordic Conference on Human-Computer Interaction · doi:10.1145/2639189.2639252 · Cited by 23 · see citing works
Designing Pronunciation Learning Tools
Robertson, S., Munteanu, C., & Penn, G. (2018) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173930 · Cited by 16 · see citing works
Obiyo, N. O., Etonyeaku, E. A. C., & Ofoegbu, T. O. (2013) · Cited by 5 · see citing works
Abbeduto, L., Ozonoff, S. J., Thurman, A. J., et al. (2014) · American Psychiatric Association Publishing eBooks · doi:10.1176/appi.books.9781585625031.rh08 · Cited by 3 · see citing works
Use of Classroom Technology to Promote Learning Among Students with Autism
Rising, K. (2017) · SUNY Digital Repository Support (State University of New York System) · Cited by 1 · see citing works
Flagged first, then by citations
Research works printed but not linked: 3
- M. J. Guralnick, 2011, "Why early intervention works: a systems perspective", Infants and young children, 24(1): 6.
- K. Tanner et al., 2010, "The Digital Technology in the Learning of Students with Autism Spectrum Disorder in Applied Classroom Settings", in Proc. of the World Conf. on Educational Media and Technology: 2586-2591.
- G. Costantini et al., 2014, "Emovo Corpus: an Italian Emotional Speech DB", in Proc. of the 9 th International Conf. on Language Resources and Evaluation Iceland, 3501-3504.
Cited by
Sorted by citations
Future directions for chatbot research: An interdisciplinary research agenda
Følstad, A., Araujo, T., Law, E. L.-C., Brandtzaeg, P. B., Papadopoulos, S., Reis, L., Baez, M., Laban, G., McAllister, P., Ischen, C., et al. (2021) · Computing · doi:10.1007/s00607-021-01016-7 · Cited by 247 · see citing works
“Chatbots have been suggested as accessible interactive systems for visually impaired in need of an easily navigable user interface [7], as conversational support for users with special needs [19], and to support youth to engage in societal issues [110].”
BackgroundSelf-citation
- Authors
- Catania, F., Di Nardo, N., Garzotto, F., & Occhiuto, D.
- Affiliations
- Politecnico di Milano (IT)
- Venue
- Proceedings of the Annual Hawaii International Conference on System Sciences
- Published
- 2019
- ISSN
- 1530-1605 (ISSN-L)
- 2572-6862
- OpenAlex
- W2931954138
- Cited by
- 17
- References
- 31
- Citations by year
- 2026: 2 · 2025: 2 · 2024: 2 · 2023: 1 · 2022: 2 · 2021: 5 · 2020: 2 · 2019: 1
- FWCI
- 3.16
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by-nc-nd
- Locations
- Hawaii International Conference on System Sciences (HICSS) (publishedVersion, cc-by-nc-nd)
- Journal of the Association for Information Systems (submittedVersion)
- BOA (University of Milano-Bicocca) (submittedVersion, other-oa)
- Virtual Community of Pathological Anatomy (University of Castilla La Mancha) (submittedVersion)
- ScholarSpace (University of Hawaii at Manoa) (submittedVersion, cc-by-nc-nd)
- Topics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Emotion and Mood Recognition
- Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Keywords
- chatbots, neurodevelopmental disorders, emotion recognition, emotional expression, harmonic characteristics, assistive technology, communication support, interdisciplinary collaboration
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
