Validity of Off-the-Shelf Automatic Speech Recognition for Assessing Speech Intelligibility and Speech Severity in Speakers With Amyotrophic Lateral Sclerosis
Open accessThe DOI matches its Crossref record; 8 of the 8 references it cites were found in a registry or an index.
Abstract
Purpose: There is increasing interest in using automatic speech recognition (ASR) systems to evaluate impairment severity or speech intelligibility in speakers with dysarthria. We assessed the clinical validity of one currently available off-the-shelf (OTS) ASR system (i.e., a Google Cloud ASR API) for indexing sentence-level speech intelligibility and impairment severity in individuals with amyotrophic lateral sclerosis (ALS), and we provided guidance for potential users of such systems in research and clinic. Method: Using speech samples collected from 52 individuals with ALS and 20 healthy control speakers, we compared word recognition rate (WRR) from the commercially available Google Cloud ASR API (Machine WRR) to clinician-provided judgments of impairment severity, as well as sentence intelligibility (Human WRR). We assessed the internal reliability of Machine and Human WRR by comparing the standard deviation of WRR across sentences to the minimally detectable change (MDC), a clinical benchmark that indicates whether results are within measurement error. We also evaluated Machine and Human WRR diagnostic accuracy for classifying speakers into clinically established categories. Results: Human WRR achieved better accuracy than Machine WRR when indexing speech severity, and, although related, Human and Machine WRR were not strongly correlated. When the speech signal was mixed with noise (noise-augmented ASR) to reduce a ceiling effect, Machine WRR performance improved. Internal reliability metrics were worse for Machine than Human WRR, particularly for typical and mildly impaired severity groups, although sentence length significantly impacted both Machine and Human WRRs. Conclusions: Results indicated that the OTS ASR system was inadequate for early detection of speech impairment and grading overall speech severity. While Machine and Human WRR were correlated, ASR should not be used as a one-to-one proxy for transcription speech intelligibility or clinician severity ratings. Overall, findings suggested that the tested OTS ASR system, Google Cloud ASR, has limited utility for grading clinical speech impairment in speakers with ALS.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Automatic speech recognition and speech variability: A review
Benzeghiba, M., De Mori, R., Deroo, O., Dupont, S., Erbes, T., Jouvet, D., Fissore, L., Laface, P., Mertins, A., Ris, C., et al. (2007) · Speech Communication · doi:10.1016/j.specom.2007.02.006 · Cited by 529
Toth, L., Hoffmann, I., Gosztolya, G., Vincze, V., Szatloczki, G., Banreti, Z., Pakaski, M., & Kalman, J. (2018) · Current Alzheimer Research · doi:10.2174/1567205014666171121114930 · Cited by 290
Dysarthria in amyotrophic lateral sclerosis: A review
Tomik, B., & Guiloff, R. J. (2010) · Amyotrophic Lateral Sclerosis · doi:10.3109/17482960802379004 · Cited by 201
Goldwater, S., Jurafsky, D., & Manning, C. D. (2010) · Speech Communication · doi:10.1016/j.specom.2009.10.001 · Cited by 166
A comparative study of adaptive, automatic recognition of disordered speech
Christensen, H., Cunningham, S., Fox, C., Green, P., & Hain, T. (2012) · Interspeech 2012 · doi:10.21437/interspeech.2012-484 · Cited by 123
Green, J. R., MacDonald, R. L., Jiang, P.-P., Cattiau, J., Heywood, R., Cave, R., Seaver, K., Ladewig, M. A., Tobin, J., Brenner, M. P., et al. (2021) · Interspeech 2021 · doi:10.21437/interspeech.2021-1384 · Cited by 74
Self-citationMustafa, M. B., Rosdi, F., Salim, S. S., & Mughal, M. U. (2015) · Expert Systems with Applications · doi:10.1016/j.eswa.2015.01.033 · Cited by 37
Automated Speech Recognition in Adult Stroke Survivors: Comparing Human and Computer Transcriptions
Jacks, A., Haley, K. L., Bishop, G., & Harmon, T. G. (2019) · Folia Phoniatrica et Logopaedica · doi:10.1159/000499156 · Cited by 30
Cited by
Automatic speech recognition of conversational speech in individuals with disordered speech
Tobin, J., Nelson, P., MacDonald, B., Heywood, R., Cave, R., Seaver, K., Desjardins, A., Jiang, P.-P., & Green, J. R. (2024) · Journal of Speech, Language, and Hearing Research · doi:10.1044/2024_jslhr-24-00045 · Cited by 45
“These results align with previous studies showing that ASR accuracy is lower for disordered speech than for typical speech and degrades as a function of severity (Gutz et al., 2022; Tomanek et al., 2023).”
Self-citationStipancic, K. L., van Brenk, F., Qiu, M., & Tjaden, K. (2025) · Journal of Speech, Language, and Hearing Research · doi:10.1044/2024_jslhr-24-00354 · Cited by 11
Self-citationThies, T., Mallick, E., Tröger, J., Baykara, E., Mücke, D., & Barbe, M. T. (2025) · npj Parkinson's Disease · doi:10.1038/s41531-025-00959-4 · Cited by 8
Carl, M., Rudyk, E., Shapira, Y., Rusiewicz, H. L., & Icht, M. (2024) · Journal of Speech, Language, and Hearing Research · doi:10.1044/2024_jslhr-24-00009 · Cited by 9
Metadata
- Authors
- Gutz, S. E., Stipancic, K. L., Yunusova, Y., Berry, J. D., & Green, J. R.
- Venue
- Journal of Speech, Language, and Hearing Research
- Published
- 2022-06-08
- Volume
- 65
- Issue
- 6
- Pages
- 2128-2143
- ISSN
- 1092-4388 (ISSN-L)
- 1558-9102
- OpenAlex
- W4281667045
- Cited by
- 32
- References
- 8
- FWCI
- 3.34
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Validity of Off-the-Shelf Automatic Speech Recognition for Assessing Speech Intelligibility and Speech Severity in Speakers With Amyotrophic Lateral Sclerosis. https://app.cochat.ai/papers/10.1044/2022_jslhr-21-00589
- The paper
- Gutz, S. E., Stipancic, K. L., Yunusova, Y., Berry, J. D., & Green, J. R. (2022). Validity of Off-the-Shelf Automatic Speech Recognition for Assessing Speech Intelligibility and Speech Severity in Speakers With Amyotrophic Lateral Sclerosis. Journal of Speech, Language, and Hearing Research. https://doi.org/10.1044/2022_jslhr-21-00589
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.