Automatic Assessment of Speech Intelligibility for Individuals With Aphasia
The DOI matches its Crossref record; 69 of the 71 references it cites were found in a registry or an index.
Abstract
Traditional in-person therapy may be difficult to access for individuals with aphasia due to the shortage of speech-language pathologists and high treatment cost. Computerized exercises offer a promising low-cost and constantly accessible supplement to in-person therapy. Unfortunately, the lack of feedback for verbal expression in existing programs hinders the applicability and effectiveness of this form of treatment. A prerequisite for producing meaningful feedback is speech intelligibility assessment. In this work, we investigate the feasibility of an automated system to assess three aspects of aphasic speech intelligibility: clarity, fluidity, and prosody. We introduce our aphasic speech corpus, which contains speech-based interaction between individuals with aphasia and a tablet-based application designed for therapeutic purposes. We present our method for eliciting reliable ground-truth labels for speech intelligibility based on the perceptual judgment of nonexpert human evaluators. We describe and analyze our feature set engineered for capturing pronunciation, rhythm, and intonation. We investigate the classification performance of our system under two conditions, one using human-labeled transcripts to drive feature extraction, and another using transcripts generated automatically. We show that some aspects of aphasic speech intelligibility can be estimated at human-level performance. Our results demonstrate the potential for the computerized treatment of aphasia and lay the groundwork for bridging the gap between human and automatic intelligibility assessment.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Generally, three types of training datasets are used for aphasia assessment tasks: (1) healthy subjects’ dataset, (2) aphasic patients’ dataset, or (3) a combination of healthy and aphasic dataset [10,12]”
From Performance Evaluation of Machine Learning Frameworks for Aphasia Assessment (2021) · cites this paper as [12]
In the citing paper
“Most research on automatic aphasia assessment [5,9,10,12,13,14,18] has focused on a single aphasia assessment task using automatic speech recognition techniques and a machine learning (ML) framework with a fixed dataset-type.”
From Performance Evaluation of Machine Learning Frameworks for Aphasia Assessment (2021) · cites this paper as [12]
In the citing paper
“The naming, repetition, and auditory verbal comprehension subtests can be automated using customized machine learning [8,11,12].”
From A Comparative Investigation of Automatic Speech Recognition Platforms for Aphasia Assessment Batteries (2023) · cites this paper as [12]
Citations
Cites
The WEKA data mining software: an update
Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., & Witten, I. H. (2009) · ACM SIGKDD Explorations Newsletter · doi:10.1145/1656274.1656278 · Cited by 17,750 · cited 1×
“Since our dataset is relatively small (1,503 data points), we did not do model selection and instead used the default settings specified in the Weka toolkit [65].”
MethodPeng, H., Long, F., & Ding, C. (2005) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2005.159 · Cited by 10,544
(2001) · Cited by 1,932
Motor Speech Disorders: Substrates, Differential Diagnosis, and Management
Duffy, J. R. (1995) · Cited by 1,892
The Geneva Minimalistic Acoustic Parameter Set (GeMAPS) for Voice Research and Affective Computing
Eyben, F., Scherer, K. R., Schuller, B. W., Sundberg, J., Andre, E., Busso, C., Devillers, L. Y., Epps, J., Laukka, P., Narayanan, S. S., et al. (2016) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2015.2457417 · Cited by 1,818 · cited 1×
“Baseline classification results using the GeMAPS acoustic feature set [53] can be found in the SI.”
MethodAcoustic Modeling Using Deep Belief Networks
Mohamed, A.-R., Dahl, G. E., & Hinton, G. (2012) · IEEE Transactions on Audio, Speech, and Language Processing · doi:10.1109/tasl.2011.2109382 · Cited by 1,727
Western Aphasia Battery--Revised
Kertesz, A. (2012) · PsycTESTS Dataset · doi:10.1037/t15168-000 · Cited by 1,367
Correlates of linguistic rhythm in the speech signal
Ramus, F., Nespor, M., & Mehler, J. (1999) · Cognition · doi:10.1016/s0010-0277(99)00058-x · Cited by 1,270
TOBI: a standard for labeling English prosody
Silverman, K., Beckman, M., Pitrelli, J., Ostendorf, M., Wightman, C., Price, P., Pierrehumbert, J., & Hirschberg, J. (1992) · 2nd International Conference on Spoken Language Processing (ICSLP 1992) · doi:10.21437/icslp.1992-260 · Cited by 1,014
Intensity of Aphasia Therapy, Impact on Recovery
Bhogal, S. K., Teasell, R., & Speechley, M. (2003) · Stroke · doi:10.1161/01.str.0000062343.64383.d0 · Cited by 719
Phone-level pronunciation scoring and assessment for interactive language learning
Witt, S. M., & Young, S. J. (2000) · Speech Communication · doi:10.1016/s0167-6393(99)00044-8 · Cited by 528
Durational variability in speech and the Rhythm Class Hypothesis
Grabe, E., & Low, E. L. (2013) · Laboratory Phonology 7 · doi:10.1515/9783110197105.515 · Cited by 527
A meta-analysis of clinical outcomes in the treatment of aphasia
Robey, R. R. (1998) · Journal of Speech, Language, and Hearing Research · doi:10.1044/jslhr.4101.172 · Cited by 506
Automatic Speech Recognition: A Deep Learning Approach
Yu, D., & Deng, L. (2015) · Signals and Communication Technology · doi:10.1007/978-1-4471-5779-3 · Cited by 474
Spoken Language Derived Measures for Detecting Mild Cognitive Impairment
Roark, B., Mitchell, M., Hosom, J.-P., Hollingshead, K., & Kaye, J. (2011) · IEEE Transactions on Audio, Speech, and Language Processing · doi:10.1109/tasl.2011.2112351 · Cited by 355
Correlates of linguistic rhythm in the speech signal
Ramus, F., Nespor, M., & Mehler, J. (2000) · Cognition · doi:10.1016/s0010-0277(00)00101-3 · Cited by 242
Edmonds, L. A., Nadeau, S. E., & Kiran, S. (2009) · Aphasiology · doi:10.1080/02687030802291339 · Cited by 228
Production deficits in aphasia: A voice-onset time analysis
Blumstein, S. E., Cooper, W. E., Goodglass, H., Statlender, S., & Gottlieb, J. (1980) · Brain and Language · doi:10.1016/0093-934x(80)90137-6 · Cited by 215
Apraxia of Speech: An overview
Ogar, J., Slama, H., Dronkers, N., Amici, S., & Luisa Gorno-Tempini, M. (2005) · Neurocase · doi:10.1080/13554790500263529 · Cited by 203
Measuring up to speech intelligibility
Miller, N. (2013) · International Journal of Language & Communication Disorders · doi:10.1111/1460-6984.12061 · Cited by 197
Computerized Script Training for Aphasia: Preliminary Results
Cherney, L. R., Halper, A. S., Holland, A. L., & Cole, R. (2008) · American Journal of Speech-Language Pathology · doi:10.1044/1058-0360(2008/003 · Cited by 196
Nicholas, L. E., & Brookshire, R. H. (1995) · Journal of Speech, Language, and Hearing Research · doi:10.1044/jshr.3801.145 · Cited by 177
Prevalence and pattern of perceived intelligibility changes in Parkinson's disease
Miller, N., Allcock, L., Jones, D., Noble, E., Hildreth, A. J., & Burn, D. J. (2007) · Journal of Neurology, Neurosurgery & Psychiatry · doi:10.1136/jnnp.2006.110171 · Cited by 177
Basso, A. (2003) · doi:10.1093/oso/9780195135879.001.0001 · Cited by 170
Speech prosody in Broca's aphasia
Danly, M., & Shapiro, B. (1982) · Brain and Language · doi:10.1016/0093-934x(82)90082-7 · Cited by 159
The Aphasia Quotient: The Taxonomic Approach to Measurement of Aphasic Disability
Kertesz, A., & Poole, E. (2004) · Canadian Journal of Neurological Sciences / Journal Canadien des Sciences Neurologiques · doi:10.1017/s0317167100120736 · Cited by 157
Breen, Dilley, Kraemer, & Gibson (2012) · Corpus Linguistics and Linguistic Theory · doi:10.1515/cllt-2012-0011 · Cited by 145
The Aphasia Quotient: The Taxonomic Approach to Measurement of Aphasic Disability
Kertesz, A., & Poole, E. (1974) · Canadian Journal of Neurological Sciences / Journal Canadien des Sciences Neurologiques · doi:10.1017/s031716710001951x · Cited by 127
Manual of Aphasia and Aphasia Therapy
Helm‐Estabrooks, N., & Albert, M. L. (2013) · Medical Entomology and Zoology · Cited by 123
Allen, L., Mehta, S., Andrew McClure, J., & Teasell, R. (2012) · Topics in Stroke Rehabilitation · doi:10.1310/tsr1906-523 · Cited by 123
A comparative study of adaptive, automatic recognition of disordered speech
Christensen, H., Cunningham, S., Fox, C., Green, P., & Hain, T. (2012) · Interspeech 2012 · doi:10.21437/interspeech.2012-484 · Cited by 123
Toward a Quantitative Basis for Assessment and Diagnosis of Apraxia of Speech
Haley, K. L., Jacks, A., de Riesthal, M., Abou-Khalil, R., & Roth, H. L. (2012) · Journal of Speech, Language, and Hearing Research · doi:10.1044/1092-4388(2012/11-0318 · Cited by 111
Direct Magnitude Estimates of Speech Intelligibility in Dysarthria: Effects of a Chosen Standard
Weismer, G., & Laures, J. S. (2002) · Journal of Speech, Language, and Hearing Research · doi:10.1044/1092-4388(2002/033 · Cited by 106
Online crowdsourcing for efficient rating of speech: A validation study
McAllister Byun, T., Halpin, P. F., & Szeredi, D. (2015) · Journal of Communication Disorders · doi:10.1016/j.jcomdis.2014.11.003 · Cited by 95
Perceptual Characteristics of Consonant Production by Apraxic Speakers
Odell, K. H., McNeil, M. R., Rosenbek, J. C., & Hunter, L. (1990) · Journal of Speech and Hearing Disorders · doi:10.1044/jshd.5502.345 · Cited by 94
Automated Intelligibility Assessment of Pathological Speech Using Phonological Features
Middag, C., Martens, J.-P., Van Nuffelen, G., & De Bodt, M. (2009) · EURASIP Journal on Advances in Signal Processing · doi:10.1155/2009/629030 · Cited by 94
Treatment intensity and childhood apraxia of speech
Namasivayam, A. K., Pukonen, M., Goshulak, D., Hard, J., Rudzicz, F., Rietveld, T., Maassen, B., Kroll, R., & van Lieshout, P. (2015) · International Journal of Language & Communication Disorders · doi:10.1111/1460-6984.12154 · Cited by 86
Speech technology‐based assessment of phoneme intelligibility in dysarthria
Nuffelen, G. V., Middag, C., Bodt, M. S. D., & Martens, J. (2008) · International Journal of Language & Communication Disorders · doi:10.1080/13682820802342062 · Cited by 85
Automatic intelligibility classification of sentence-level pathological speech
Kim, J., Kumar, N., Tsiartas, A., Li, M., & Narayanan, S. S. (2015) · Computer Speech & Language · doi:10.1016/j.csl.2014.02.001 · Cited by 83
Automatic word naming recognition for an on-line aphasia treatment system
Abad, A., Pompili, A., Costa, Â., Trancoso, I. M., Fonseca, J., Leal, G., Farrajota, L., & Martins, I. P. (2012) · Computer Speech & Language · doi:10.1016/j.csl.2012.10.003 · Cited by 82
Integral and Diagnostic Intrusive Prediction of Speech Quality
Côté, N. (2011) · T-labs series in telecommunication services · doi:10.1007/978-3-642-18463-5 · Cited by 74
Patient-Reported Changes in Communication After Computer-Based Script Training for Aphasia
Manheim, L. M., Halper, A. S., & Cherney, L. R. (2009) · Archives of Physical Medicine and Rehabilitation · doi:10.1016/j.apmr.2008.10.022 · Cited by 64
Pakhomov, S. V. S., Smith, G. E., Chacon, D., Feliciano, Y., Graff-Radford, N., Caselli, R., & Knopman, D. S. (2010) · Cognitive and Behavioral Neurology · doi:10.1097/wnn.0b013e3181c5dde3 · Cited by 64 · cited 1×
“4 s) count [24], phonation rate, mean [54], median, minimum, maximum, and standard deviation of pause durations.”
BackgroundCombining in-domain and out-of-domain speech data for automatic recognition of disordered speech
Christensen, H., Aniol, M. B., Bell, P., Green, P. D., Hain, T., King, S., & Swietojanski, P. (2013) · Interspeech 2013 · doi:10.21437/interspeech.2013-324 · Cited by 63
Using text and acoustic features to diagnose progressive aphasia and its subtypes
Fraser, K. C., Rudzicz, F., & Rochon, E. (2013) · Interspeech 2013 · doi:10.21437/interspeech.2013-514 · Cited by 61
Computer-based script training for aphasia: Emerging themes from post-treatment interviews
Cherney, L. R., Halper, A. S., & Kaye, R. C. (2011) · Journal of Communication Disorders · doi:10.1016/j.jcomdis.2011.04.002 · Cited by 58 · cited 1×
“than word-level exercises for recovering communication skills in highly routine conversational tasks [7], [49].”
BackgroundComparing Humans and Automatic Speech Recognition Systems in Recognizing Dysarthric Speech
Mengistu, K. T., & Rudzicz, F. (2011) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-21043-3_36 · Cited by 55
Landa, S., Pennington, L., Miller, N., Robson, S., Thompson, V. B., & Steen, N. (2014) · International Journal of Speech-Language Pathology · doi:10.3109/17549507.2014.927922 · Cited by 54
Dysprosody in Broca's aphasia: A case study*1
Gandour, J. (1989) · Brain and Language · doi:10.1016/0093-934x(89)90017-5 · Cited by 48
Automatic Prediction of Children's Reading Ability for High-Level Literacy Assessment
Black, M. P., Tepperman, J., & Narayanan, S. S. (2011) · IEEE Transactions on Audio, Speech, and Language Processing · doi:10.1109/tasl.2010.2076389 · Cited by 48
Telediagnostic assessment of intelligibility in dysarthria: A pilot investigation of MVP-online
Ziegler, W., & Zierdt, A. (2008) · Journal of Communication Disorders · doi:10.1016/j.jcomdis.2008.05.001 · Cited by 46
Learning diagnostic models using speech and language measures
Peintner, B., Jarrold, W., Vergyri, D., Richey, C., Tempini, M. L. G., & Ogar, J. (2008) · 2008 30th Annual International Conference of the IEEE Engineering in Medicine and Biology Society · doi:10.1109/iembs.2008.4650249 · Cited by 41
Mustafa, M. B., Rosdi, F., Salim, S. S., & Mughal, M. U. (2015) · Expert Systems with Applications · doi:10.1016/j.eswa.2015.01.033 · Cited by 37
Computers in the Treatment of Chronic Aphasia
Katz, R. (2010) · Seminars in Speech and Language · doi:10.1055/s-0029-1244951 · Cited by 34
Supervised Home Training of Dialogue Skills in Chronic Aphasia: A Randomized Parallel Group Study
Nobis-Bosch, R., Springer, L., Radermacher, I., & Huber, W. (2011) · Journal of Speech, Language, and Hearing Research · doi:10.1044/1092-4388(2010/09-0204 · Cited by 31
Automatic scoring of the intelligibility in patients with cancer of the oral cavity
Maier, A., Schuster, M. E., Batliner, A. M., Nöth, E., & Nkenke, E. (2007) · Interspeech (USB) · doi:10.21437/interspeech.2007-388 · Cited by 27
Christensen, H., Casanueva, I., Cunningham, S. P., Green, P., & Hain, T. (2014) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt.2014.7078583 · Cited by 27
Automatic analysis of speech quality for aphasia treatment
Le, D., Licata, K., Mercado, E., Persad, C., & Provost, E. M. (2014) · 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2014.6854524 · Cited by 23
Self-citationLe, D., & Provost, E. M. (2014) · Interspeech (USB) · doi:10.21437/interspeech.2014-373 · Cited by 23
Self-citationModeling pathological speech perception from data with similarity labels
Berisha, V., Liss, J., Sandoval, S., Utianski, R. L., & Spanias, A. (2014) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2014.6853730 · Cited by 22
Using dynamic time warping to compute prosodic similarity measures
Rilliard, A. O. B., Allauzen, A., & de Mareüil, P. B. (2011) · Interspeech (USB) · doi:10.21437/interspeech.2011-531 · Cited by 22
Learning speaker-specific pronunciations of disordered speech
Christensen, H., Green, P. D., & Hain, T. (2013) · Interspeech 2013 · doi:10.21437/interspeech.2013-315 · Cited by 21
Temporal and spectral properties of voiceless fricatives in aphasia and apraxia of speech
Haley, K. L. (2002) · Aphasiology · doi:10.1080/02687030244000257 · Cited by 21
Jellinger, K. A. (2003) · European Journal of Neurology · doi:10.1046/j.1468-1331.2003.00603_2.x · Cited by 20
Towards robust automatic evaluation of pathologic telephone speech
Riedhammer, K., Stemmer, G., Haderlein, T., Schuster, M. E., Rosanowski, F., Nöth, E., & Maier, A. (2007) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru.2007.4430200 · Cited by 18
Testing suprasegmental English through parroting
Tepperman, J., Stanley, T., Hacıoğlu, K., & Pellom, B. L. (2010) · Speech prosody · doi:10.21437/speechprosody.2010-73 · Cited by 17
A Critical Review of Acoustic Analyses of Aphasic and/or Apraxic Speech
Wambaugh, J. L., Doyle, P. J., Kalinyak-Fliszar, M., & West, J. E. (1996) · The Aphasiology Archive (University of Pittsburgh) · Cited by 8
Attention shift decoding for conversational speech recognition
Kumaran, R. S., Bilmes, J., & Kirchhoff, K. (2007) · Interspeech (USB) · doi:10.21437/interspeech.2007-432 · Cited by 7
Continuous speech recognition using attention shift decoding with soft decision
Kalinli, O., & Narayanan, S. S. (2009) · Interspeech (USB) · doi:10.21437/interspeech.2009-557 · Cited by 4
Research works printed but not linked: 2
- davis, 2006, Aphasiology Disorders and clinical practice
- wambaugh, 2006, Treatment guidelines for acquired apraxia of speech: A synthesis and evaluation of the evidence, J Med Speech-Lang Pathol, 14, 15
Cited by
Automatic quantitative analysis of spontaneous aphasic speech
Le, D., Licata, K., & Mower Provost, E. (2018) · Speech Communication · doi:10.1016/j.specom.2018.04.001 · Cited by 97
“Features 1–2 capture a PWA’s speech rate, which has been shown in previous work to be useful for assessing the quality of aphasic speech (Le et al., 2014; Le and Provost, 2014; Le et al., 2016) as well as distinguishing between subjects with PPA and healthy controls (Fraser et al.”
BackgroundMethodInfluentialSelf-citationGhasemzadeh, H., Hillman, R. E., & Mehta, D. D. (2024) · Journal of Speech, Language, and Hearing Research · doi:10.1044/2023_jslhr-23-00273 · Cited by 72
“…their cry (Matikolaie & Tadj, 2022), abnormal vocal folds from laryngoscopic images (Cho & Choi, 2020), intelligibility assessment of aphasic speech (Le et al., 2016), swallowing disorder from high-resolution manometry recordings (Mielens et al., 2012), screening for autism spectrum disorder…”
BackgroundLauraitis, A., Maskeliūnas, R., Damaševičius, R., & Krilavičius, T. (2020) · IEEE Access · doi:10.1109/access.2020.2995737 · Cited by 66
“Perez et al. [41] differentiate between healthy controls and HD patients) based on acoustic and lexical features (MFCC, GMM, pause, speech rate, goodness of Pronunciation (GoP) [42]).”
BackgroundClassification of Huntington Disease Using Acoustic and Lexical Features
Perez, M. C., Jin, W., Le, D., Carlozzi, N. E., Dayalu, P., Roberts, A., & Provost, E. M. (2018) · Interspeech (USB) · doi:10.21437/interspeech.2018-2029 · Cited by 60
“trained an ASR system on aphasic speech and used this system to extract speech features for the prediction of intelligibility, including: the number of fillers, phone rate, error rate, utterance length, and goodness of pronunciation [11].”
BackgroundMethodSelf-citationFernández-Díaz, M., & Gallardo-Antolín, A. (2020) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2020.103976 · Cited by 50
Metadata
- Authors
- Le, D., Licata, K., Persad, C., & Provost, E. M.
- Affiliations
- University of Michigan (US)
- Venue
- IEEE/ACM Transactions on Audio, Speech, and Language Processing
- Published
- 2016-11
- Volume
- 24
- Issue
- 11
- Pages
- 2187-2199
- ISSN
- 2329-9290 (ISSN-L)
- 2329-9304
- OpenAlex
- W2502847729
- Cited by
- 59
- References
- 71
- Citations by year
- 2026: 5 · 2025: 8 · 2024: 4 · 2023: 6 · 2022: 3 · 2021: 5 · 2020: 10 · 2019: 7 · 2018: 7 · 2017: 4
- FWCI
- 4.93
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- IEEE/ACM Transactions on Audio Speech and Language Processing (publishedVersion)
- Funders
- University of Michigan
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Health Sciences › Health Professions › Speech and Hearing › Voice and Speech Disorders
- Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
- Keywords
- aphasia, speech intelligibility assessment, speech clarity, prosody, intonation, rhythm, feature engineering, ground truth annotation, perceptual judgment
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.