Assessing Virtual Assistant Capabilities with Italian Dysarthric Speech
Open accessThe DOI matches its Crossref record; 12 of the 12 references it cites were found in a registry or an index.
Abstract
The usage of smartphone-based virtual assistants (e.g., Siri or Google Assistant) is growing, and their spread has generally a positive impact on device accessibility, e.g., for people with disabilities. However, people with dysarthria or other speech impairments may be unable to use these virtual assistants with proficiency. This paper investigates to which extent people with ALS-induced dysarthria can be understood and get consistent answers by three widely used smartphone-based assistants, namely Siri, Google Assistant, and Cortana. We focus on the recognition of Italian dysarthric speech, to study the behavior of the virtual assistants with this specific population for which no relevant studies are available. We collected and recorded suitable speech samples from people with dysarthria in a dedicated center of the Molinette hospital, in Turin, Italy. Starting from those recordings, the differences between such assistants, in terms of speech recognition and consistency in answer, are investigated and discussed. Results highlight different performance among the virtual assistants. For speech recognition, Google Assistant is the most promising, with around 25% of word error rate per sentence. Consistency in answer, instead, sees Siri and Google Assistant provide coherent answers around 60% of times.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Speech and cognitive skills play a significant role in the ability to effectively control voice assistants [17,19,22].”
From Investigating the Accessibility of Voice Assistants With Impaired Users: Mixed Methods Study (2020) · cites this paper as [19]
In the citing paper
“The accessibility of voice assistants has not been thoroughly investigated yet.”
From Investigating the Accessibility of Voice Assistants With Impaired Users: Mixed Methods Study (2020) · cites this paper as [19]
In the citing paper
“studies evaluating the accessibility of such systems are still sparse and fail to account for the participants’ actual motor, linguistic, and cognitive abilities.”
From Investigating the Accessibility of Voice Assistants With Impaired Users: Mixed Methods Study (2020) · cites this paper as [19]
Citations
Cites
Cedarbaum, J. M., Stambler, N., Malta, E., Fuller, C., Hilt, D., Thurmond, B., & Nakanishi, A. (1999) · Journal of the Neurological Sciences · doi:10.1016/s0022-510x(99)00210-5 · Cited by 3,307
Motor Speech Disorders: Substrates, Differential Diagnosis, and Management
Duffy, J. R. (1995) · Cited by 1,892
López, G., Quesada, L., & Guerrero, L. A. (2018) · Advances in Intelligent Systems and Computing · doi:10.1007/978-3-319-60366-7_23 · Cited by 277
Dysarthria in amyotrophic lateral sclerosis: A review
Tomik, B., & Guiloff, R. J. (2010) · Amyotrophic Lateral Sclerosis · doi:10.3109/17482960802379004 · Cited by 201
Speech Recognition as an Input to Electronic Assistive Technology
Hawley, M. (2002) · British Journal of Occupational Therapy · doi:10.1177/030802260206500104 · Cited by 47 · cited 1×
“Hawley [7] presents an early overview, based on a literature review and clinical observations, upon the suitability and performance of speech recognition for computer access by people with disabilities, including people with dysarthria.”
BackgroundUsing articulatory likelihoods in the recognition of dysarthric speech
Rudzicz, F. (2011) · Speech Communication · doi:10.1016/j.specom.2011.10.006 · Cited by 46 · cited 1×
“To overcome this kind of issues, researchers investigated new methods and proposed dedicated ASR systems for dysarthria, e.g., by using ergodic hidden Markov models [12] or articulatory dynamic Bayes networks [13].”
MethodPolur, P. D., & Miller, G. E. (2005) · Medical Engineering & Physics · doi:10.1016/j.medengphy.2005.11.002 · Cited by 45 · cited 1×
“To overcome this kind of issues, researchers investigated new methods and proposed dedicated ASR systems for dysarthria, e.g., by using ergodic hidden Markov models [12] or articulatory dynamic Bayes networks [13].”
MethodOn How Deaf People Might Use Speech to Control Devices
Bigham, J. P., Kushalnagar, R., Huang, T.-H. K., Flores, J. P., & Savage, S. (2017) · Proceedings of the 19th International ACM SIGACCESS Conference on Computers and Accessibility · doi:10.1145/3132525.3134821 · Cited by 34 · cited 2×
“For example, Bigham et al. [2] demonstrated that Google’s speech recognition system does not work well for people who are deaf or hard of hearing, and they expected that recognizing deaf speech will remain challenging for both automatic and human-powered approaches.”
BackgroundDesigning voice interaction for people with physical and speech impairments
Derboven, J., Huyghe, J., & Grooff, D. D. (2014) · Nordic Conference on Human-Computer Interaction · doi:10.1145/2639189.2639252 · Cited by 29
Glasser, A. T., Kushalnagar, K. R., & Kushalnagar, R. S. (2017) · Proceedings of the 19th International ACM SIGACCESS Conference on Computers and Accessibility · doi:10.1145/3132525.3134819 · Cited by 28 · cited 1×
“Glasser et al. [6], instead, focus on the issues that may arise from the usage of two virtual assistants by people who are deaf and hard of hearing.”
Background"Hey Siri, do you understand me?": Virtual Assistants and Dysarthria
Ballati, F., Corno, F., & Russis, L. D. (2018) · PORTO@Iris · doi:10.3233/978-1-61499-874-7-557 · Cited by 20 · cited 1×
“Ballati et al. [1], instead, investigated the interaction of English dysarthric speech data with three widely used virtual assistants, included in several standalone and mobile devices (Apple’s Siri, Google Assistant, and Amazon Alexa).”
MethodSelf-citationWorkshop Proceedings of the 19th International Conference on Intelligent Environments (IE2023)
Online, I. C. O. I. E. 2. U., Bekaroo, G., Allouch, S. B., & Mecella, M. (2023) · Ambient intelligence and smart environments · doi:10.3233/aise32 · Cited by 4
Research works printed but not linked: 1
- Therapy Development Institute ALS. 2018 . ALS Frequently Asked Questions . Retrieved June 13, 2018 from https://www.als.net/about-als-tdi/als-faq/. Therapy Development Institute ALS. 2018. ALS Frequently Asked Questions. Retrieved June 13, 2018 from https://www.als.net/about-als-tdi/als-faq/.
Cited by
Investigating the Accessibility of Voice Assistants With Impaired Users: Mixed Methods Study
Masina, F., Orso, V., Pluchino, P., Dainese, G., Volpato, S., Nelini, C., Mapelli, D., Spagnolli, A., & Gamberini, L. (2020) · Journal of Medical Internet Research · doi:10.2196/18431 · Cited by 105
“First, the actual speech abilities of the users were not assessed because they were either self-reported [7] or not reported at all [17,19].”
BackgroundResultInfluentialOn the impact of dysarthric speech on contemporary ASR cloud platforms
Russis, L. D., & Corno, F. (2019) · Journal of Reliable Intelligent Environments · doi:10.1007/s40860-019-00085-y · Cited by 66
“, [1,2]) and with other voice impairments (e.”
BackgroundSelf-citationInteraction between people with dysarthria and speech recognition systems: A review
Jaddoh, A., Loizides, F., & RANA, O. F. (2022) · Assistive Technology · doi:10.1080/10400435.2022.2061085 · Cited by 45
“Two more classifications derived from the reviewed literature are commercial systems (Ballati et al., 2018a, 2018b; De Russis & Corno, 2019; Moore et al., 2018) and non-commercial systems.”
InfluentialA Systematic Review of Ethical Concerns with Voice Assistants
Seymour, W., Zhan, X., Coté, M., & Such, J. M. (2023) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3600211.3604679 · Cited by 68
“While requests for information about vaccines were handled reasonably well by Google Assistant and Siri, Alexa understood fewer queries and was less likely to present information from authoritative sources [127].”
BackgroundMethodInfluentialBrahmi, Z., Mahyoob, M., Al-Sarem, M., Algaraady, J., Bousselmi, K., & Alblwi, A. (2024) · Psychology Research and Behavior Management · doi:10.2147/prbm.s460283 · Cited by 36
“[13,15,55,70,72,80,87,94,95,97–99,101,112,117,127,131,133,134] Spectro-Temporal of utterances/keywords 26.15% [12,14,15,26,63,72,87,89,90,97,101–103,110,112,124,136] Articulation way/Speech timing 18.5% [13,27,58,81,86–88,111,113,124,129] Glottal flow 7.7% [58,72,73,87,132] Tongue…”
Influential
Metadata
- Authors
- Ballati, F., Corno, F., & De Russis, L.
- Affiliations
- Politecnico di Torino (IT)
- Politecnico di Torino, Torino, Italy
- Venue
- Proceedings of the 20th International ACM SIGACCESS Conference on Computers and Accessibility
- Publisher
- ACM
- Published
- 2018-10-08
- Pages
- 93-101
- OpenAlex
- W2896133140
- Cited by
- 39
- References
- 12
- Citations by year
- 2026: 2 · 2025: 4 · 2024: 6 · 2023: 6 · 2022: 4 · 2021: 13 · 2020: 3 · 2019: 1
- FWCI
- 1.73
- Type
- conference paper
- Language
- en
- Open access
- green
- hdl.handle.net
- Locations
- International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) (publishedVersion)
- PORTO@Iris (submittedVersion, other-oa)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Health Sciences › Health Professions › Speech and Hearing › Voice and Speech Disorders
- Health Sciences › Health Professions › Occupational Therapy › Assistive Technology in Communication and Mobility
- Keywords
- virtual assistants, dysarthria, amyotrophic lateral sclerosis, speech disorders, speech recognition, Italian speech, Siri, Google Assistant, Cortana, assistive technology, word error rate, answer consistency, smartphone accessibility
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.