A Comparative Investigation of Automatic Speech Recognition Platforms for Aphasia Assessment Batteries
Open accessThe DOI matches its Crossref record; 29 of the 29 references it cites were found in a registry or an index; none of the 17 sources its claims cite is checked for support yet.
Abstract
The rehabilitation of aphasics is fundamentally based on the assessment of speech impairment. Developing methods for assessing speech impairment automatically is important due to the growing number of stroke cases each year. Traditionally, aphasia is assessed manually using one of the well-known assessment batteries, such as the Western Aphasia Battery (WAB), the Chinese Rehabilitation Research Center Aphasia Examination (CRRCAE), and the Boston Diagnostic Aphasia Examination (BDAE). In aphasia testing, a speech-language pathologist (SLP) administers multiple subtests to assess people with aphasia (PWA). The traditional assessment is a resource-intensive process that requires the presence of an SLP. Thus, automating the assessment of aphasia is essential. This paper evaluated and compared custom machine learning (ML) speech recognition algorithms against off-the-shelf platforms using healthy and aphasic speech datasets on the naming and repetition subtests of the aphasia battery. Convolutional neural networks (CNN) and linear discriminant analysis (LDA) are the customized ML algorithms, while Microsoft Azure and Google speech recognition are off-the-shelf platforms. The results of this study demonstrated that CNN-based speech recognition algorithms outperform LDA and off-the-shelf platforms. The ResNet-50 architecture of CNN yielded an accuracy of 99.64 ± 0.26% on the healthy dataset. Even though Microsoft Azure was not trained on the same healthy dataset, it still generated comparable results to the LDA and superior results to Google’s speech recognition platform.
Overview
CoChat Summary
## TL;DR
This paper compares two custom speech-recognition models (CNN and LDA) with Microsoft Azure and Google speech-to-text for aphasia assessment on Mandarin naming and repetition tasks. Across three training/testing scenarios, the ResNet-50 CNN performed best, and Microsoft Azure outperformed Google among the off-the-shelf platforms.
## Key findings
- On the healthy-only dataset (20 Mandarin words, 20 classes), ResNet-50 CNN with HTD time-frequency images achieved 99.64 ± 0.26% accuracy, versus 95.28 ± 0.79% for LDA; Microsoft Azure was slightly over 88%, and Google was below 75%.
- On the joint healthy-training / aphasic-testing dataset (20 common Mandarin words), CNN reached 59.17 ± 0.003% accuracy, versus 54.34 ± 0.79% for LDA; both off-the-shelf platforms scored below 31%.
- On the aphasic-only dataset, CNN achieved 67.78 ± 0.003% accuracy, versus 45.63 ± 0.79% for LDA; Microsoft Azure and Google were not included because they were pre-trained on healthy speech.
- For healthy participants’ CNN architecture comparison, VGG16 had the highest accuracy and F1-score on the healthy-only words dataset (99.75 ± 0.1%), while ResNet-50 was best on healthy vowels + words (98.18 ± 0.57%) and was then used for the main comparison.
## Methods
- The study used speech recorded from 12 aphasic patients (including 5 females, mean age 61.8 ± 14.4) and 34 healthy subjects (including 11 females, mean age 21.5 ± 3.1), all speaking Mandarin; the dataset included 20 words for the main analyses, and 6 vowels in some CNN comparisons.
- Three scenarios were tested: (1) training/testing on healthy speech, (2) training on healthy speech and testing on aphasic speech, and (3) training/testing on aphasic speech; evaluation used accuracy, precision, recall, and F1-score.
- Custom models were CNN (ResNet-50 and other architectures) and LDA; CNNs used HTD time-frequency images, transfer learning, five-fold cross-validation, and Wilcoxon signed-rank tests for statistical significance.
## Limitations
- The paper states that aphasic speech datasets are scarce and often small, with multiple aphasia types and severity levels; it also notes data scarcity, abnormal speech patterns, and speaker variability as challenges.
- The comparison is explicitly not absolute because the custom models were trained on a small set of isolated words, while the off-the-shelf platforms were trained on a very large dataset comprising isolated and continuous speech.
- Evident from the text: the main comparison is restricted to Mandarin naming and repetition subtests using isolated, predefined words, so the results may not extend to spontaneous speech or other aphasia battery subtests.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures




Verification
Claims the paper makes
0 Valid · 1 Not valid · 11 Not checked · 30 Own results and interpretations
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Even though Microsoft Azure was not trained on the same healthy dataset, it still generated comparable results to the LDA and superior results to Google’s speech recognition platform.”
- Also stated in: § 5.1. The Healthy Dataset Scenario · boosted
“Clearly, Microsoft Azure displayed a superior performance.”
§ A Comparative Investigation of Automatic Speech Recognition Platforms for Aphasia Assessment Batteries > Abstract The rehabilitation of aphasics is fundamentally based on the assessment of speech impairment. Developing methods for assessing speech impairment automatically is important due to the growing number of stroke cases each year. Traditionally, aphasia is assessed manually using one of the well-known assessment batteries, such as the Western Aphasia Battery (WAB), the Chinese Rehabilitation Research Center Aphasia Examination (CRRCAE), and the Boston Diagnostic Aphasia Examination (BDAE). In aphasia testing, a speech-language pathologist (SLP) administers multiple subtests to assess people with aphasia (PWA). The traditional assessment is a resource-intensive process that requires the presence of an SLP. Thus, automating the assessment of aphasia is essential. This paper evaluated and compared custom machine learning (ML) speech recognition algorithms against off-the-shelf platforms using healthy and aphasic speech datasets on the naming and repetition subtests of the aphasia battery. Convolutional neural networks (CNN) and linear discriminant analysis (LDA) are the customized ML algorithms, while Microsoft Azure and Google speech recognition are off-the-shelf platforms. The results of this study demonstrated that CNN-based speech recognition algorithms outperform LDA and off-the-shelf platforms. The ResNet-50 architecture of CNN yielded an accuracy of 99.64 ± 0.26% on the healthy dataset. Even though Microsoft Azure was not trained on the same healthy dataset, it still generated comparable results to the LDA and superior results to Google’s speech recognition platform.
boosted- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § 5.1. The Healthy Dataset Scenario · boosted
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The VGG16 model showed the highest accuracy and F1-score, i.e., 99.75 ± 0.1%, for the healthy participants’ only words dataset.”
§ A Comparative Investigation of Automatic Speech Recognition Platforms for Aphasia Assessment Batteries > 3. Materials and Methods > 3.4. Deep Neural Network Framework
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For the vowels + words healthy participants’ dataset, ResNet-50 was observed to have a statistically significantly (p < 0.05) higher accuracy and F1-score than that of VGG16.”
§ A Comparative Investigation of Automatic Speech Recognition Platforms for Aphasia Assessment Batteries > 3. Materials and Methods > 3.4. Deep Neural Network Framework
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The ResNet-50 architecture of CNN yielded an accuracy of 99.64 ± 0.26% on the healthy dataset.”
- Also stated in: § 4.1. Machine Learning Algorithms Performance on Healthy Dataset · plain
“In terms of accuracy, the CNN algorithm hit 99.64 ± 0.26%, whereas the LDA scored a lower 95.28 ± 0.79%.”
§ A Comparative Investigation of Automatic Speech Recognition Platforms for Aphasia Assessment Batteries > Abstract The rehabilitation of aphasics is fundamentally based on the assessment of speech impairment. Developing methods for assessing speech impairment automatically is important due to the growing number of stroke cases each year. Traditionally, aphasia is assessed manually using one of the well-known assessment batteries, such as the Western Aphasia Battery (WAB), the Chinese Rehabilitation Research Center Aphasia Examination (CRRCAE), and the Boston Diagnostic Aphasia Examination (BDAE). In aphasia testing, a speech-language pathologist (SLP) administers multiple subtests to assess people with aphasia (PWA). The traditional assessment is a resource-intensive process that requires the presence of an SLP. Thus, automating the assessment of aphasia is essential. This paper evaluated and compared custom machine learning (ML) speech recognition algorithms against off-the-shelf platforms using healthy and aphasic speech datasets on the naming and repetition subtests of the aphasia battery. Convolutional neural networks (CNN) and linear discriminant analysis (LDA) are the customized ML algorithms, while Microsoft Azure and Google speech recognition are off-the-shelf platforms. The results of this study demonstrated that CNN-based speech recognition algorithms outperform LDA and off-the-shelf platforms. The ResNet-50 architecture of CNN yielded an accuracy of 99.64 ± 0.26% on the healthy dataset. Even though Microsoft Azure was not trained on the same healthy dataset, it still generated comparable results to the LDA and superior results to Google’s speech recognition platform.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § 4.1. Machine Learning Algorithms Performance on Healthy Dataset · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In terms of accuracy, the CNN algorithm hit 99.64 ± 0.26%, whereas the LDA scored a lower 95.28 ± 0.79%.”
§ A Comparative Investigation of Automatic Speech Recognition Platforms for Aphasia Assessment Batteries > 4. Results > 4.1. Machine Learning Algorithms Performance on Healthy Dataset
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
20 more claims
Create a free account to see all 42 claims and to verify the 17 references still open.
- It may state two separate claims as one. 12
- Probably not a substantial claim. 11
- The quote only partly states this claim. 2
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The analysis results showed that the machine learning algorithms used in post-stroke aphasia models were mainly supervised methods, including random forests, neural networks, and support vector machines.”
From Machine learning models in post-stroke aphasia: a scoping review (2026) · cites this paper as [27]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“three studies (27, 31, 35) automatically assessed speech intelligibility or identified emotional states of patients based on acoustic or time-frequency features of speech.”
From Machine learning models in post-stroke aphasia: a scoping review (2026) · cites this paper as [27]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Seven studies (19–21, 24, 27, 29, 31) used multiple algorithms to construct models and compare them.”
From Machine learning models in post-stroke aphasia: a scoping review (2026) · cites this paper as [27]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Twelve studies (19, 20, 22, 24, 25, 27–30, 33, 34, 37) mainly used accuracy, AUC, F1-score, precision, recall, and other indicators.”
From Machine learning models in post-stroke aphasia: a scoping review (2026) · cites this paper as [27]
The cited source says what the claim attributes to it.
In the citing paper
“Some studies (22, 27, 31) also included speech or behavioral data of healthy subjects as a control or training basis.”
From Machine learning models in post-stroke aphasia: a scoping review (2026) · cites this paper as [27]
7 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 1×
“Considering the characteristics of the ImageNet dataset [38], all TFD RGB color images were resized to 224 × 224 × 3 pixels before feeding them to the pre-trained CNN models.”
Cyclical Learning Rates for Training Neural Networks
Smith, L. N. (2017) · 2017 IEEE Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv.2017.58 · Cited by 2,670 · cited 1×
“003 was used for fine-tuning the pre-trained models using the TFD image datasets [39].”
Feigin, V. L., Norrving, B., & Mensah, G. A. (2017) · Circulation Research · doi:10.1161/circresaha.116.308413 · Cited by 2,184 · cited 1×
“One recent report stated that over 80 million people suffer strokes worldwide, and one-third of them have aphasia [2].”
(1992) · Cited by 1,405
Understanding how image quality affects deep neural networks
Dodge, S., & Karam, L. (2016) · 2016 Eighth International Conference on Quality of Multimedia Experience (QoMEX) · doi:10.1109/qomex.2016.7498955 · Cited by 835 · cited 1×
“The HTD has been found to produce a high-resolution TF image of Mandarin speech signals; hence, it can improve speech signal classification when used with the CNN model [11,36,37].”
Novel Speech Signal Processing Algorithms for High-Accuracy Classification of Parkinson’s Disease
Tsanas, A., Little, M. A., McSharry, P. E., Spielman, J., & Ramig, L. O. (2012) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2012.2183367 · Cited by 732 · cited 2×
“, 2012 [17] CML: Support Vector Machines (SVM), random forests • This work investigated how accurately speech signal processing algorithms (dysphonia measures) can predict Parkinson’s disease (PD) symptom severity using speech signals.”
The global burden of stroke: persistent and disabling
Gorelick, P. B. (2019) · The Lancet Neurology · doi:10.1016/s1474-4422(19)30030-4 · Cited by 616
A meta-analysis of clinical outcomes in the treatment of aphasia
Robey, R. R. (1998) · Journal of Speech, Language, and Hearing Research · doi:10.1044/jslhr.4101.172 · Cited by 506 · cited 1×
“Previous studies have indicated that the early commencement of supervised rehabilitation of PWA leads to a faster recovery [3].”
Huber, W., Poeck, K., & Willmes, K. (1984) · PubMed · Cited by 338
Voice Disorder Identification by Using Machine Learning Techniques
Verde, L., De Pietro, G., & Sannino, G. (2018) · IEEE Access · doi:10.1109/access.2018.2816338 · Cited by 156 · cited 1×
“It is also possible to discriminate between healthy and aphasic speech (binary classification problem) by adding a decision logic associated with a cut-off threshold at the output of the classifiers [9,46].”
Research works printed but not linked: 3
- Ding, 1995, Jisuanji fuzhu yanyu jiaozhi xitong. [A computer aided speech correction system], Zhongguo Shengwu Yixue Gongcheng Xuebao, 14, 39
- Li, A.N. (2010). Shiyuzheng Huanzhe Yuyin Xinhao de Shibie Yanjiu. An Investigation on Speech Recognition of Aphasia Patients. [Master’s Thesis, Xi’an University of Science and Technology].
- Kingma, D., and Adam, J.B. (2014). A Method for Stochastic Optimization. arXiv.
Cited by
Deep learning on medical image analysis
Wang, J., Wang, S., & Zhang, Y. (2024) · CAAI Transactions on Intelligence Technology · doi:10.1049/cit2.12356 · Cited by 162
Zhang, M., Tang, E., Ding, H., & Zhang, Y. (2024) · Journal of Speech, Language, and Hearing Research · doi:10.1044/2024_jslhr-24-00157 · Cited by 22
LLM-Based Text Prediction and Question Answer Models for Aphasia Speech
Manir, S. B., Islam, K. M. S., Madiraju, P., & Deshpande, P. (2024) · IEEE Access · doi:10.1109/access.2024.3443592 · Cited by 10
“The research in question highlights the capacity of customized machine learning methods to enhance the accuracy of recognizing speech in individuals with aphasia [18].”
BackgroundA Systematic Review of Using Deep Learning in Aphasia: Challenges and Future Directions
Wang, Y., Cheng, W., Sufi, F. K., Fang, Q., & Mahmoud, S. S. (2024) · Computers · doi:10.3390/computers13050117 · Cited by 17
Self-citationRemya, M. S., Raman, R., Sankaran, R., Namboodiri, V., & Nedungadi, P. (2025) · IEEE Access · doi:10.1109/access.2025.3620114 · Cited by 18
“Deep learning architecture, including deep neural networks (DNNs) [92], convolutional neural networks (CNNs) [93], [94], and recurrent neural networks (RNNs), dominate speech and language disorder classification.”
Metadata
- Authors
- Mahmoud, S. S., Pallaud, R. F., Kumar, A., Faisal, S., Wang, Y., & Fang, Q.
- Affiliations
- Shantou University (CN)
- Phoenix College (US)
- Corresponding author
- Mahmoud, Seedahmed S.
- Fang, Qiang
- Venue
- Sensors
- Publisher
- MDPI AG
- Published
- 2023-01-11
- Volume
- 23
- Issue
- 2
- Pages
- 857
- ISSN
- 1424-8220 (ISSN-L)
- PMC
- PMC9863375
- OpenAlex
- W4315786975
- Cited by
- 25
- References
- 29
- Citations by year
- 2026: 5 · 2025: 7 · 2024: 10 · 2023: 3
- FWCI
- 2.36
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Sensors (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Li Ka Shing Foundation (2020LKSFG04C)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Health Sciences › Health Professions › Speech and Hearing › Voice and Speech Disorders
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neurobiology of Language and Bilingualism
- Keywords
- aphasia assessment, speech recognition, convolutional neural networks, ResNet50, linear discriminant analysis, stroke rehabilitation, Western Aphasia Battery, Boston Diagnostic Aphasia Examination
- MeSH terms
- Aphasia, Humans, Language, Speech, Speech Disorders, Speech Perception, Stroke
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-11, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-11 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: A Comparative Investigation of Automatic Speech Recognition Platforms for Aphasia Assessment Batteries. https://app.cochat.ai/papers/10.3390/s23020857
- The paper
- Mahmoud, S. S., Pallaud, R. F., Kumar, A., Faisal, S., Wang, Y., & Fang, Q. (2023). A Comparative Investigation of Automatic Speech Recognition Platforms for Aphasia Assessment Batteries. Sensors. https://doi.org/10.3390/s23020857
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.