An End-to-End Approach to Automatic Speech Assessment for Cantonese-speaking People with Aphasia
Open accessThe DOI matches its Crossref record; 42 of the 42 references it cites were found in a registry or an index.
Abstract
Conventional automatic assessment of pathological speech usually follows two main steps: (1) extraction of pathology-specific features; (2) classification or regression on extracted features. Given the great variety of speech and language disorders, feature design is never a straightforward task, and yet it is most crucial to the performance of assessment. This paper presents an end-to-end approach to automatic speech assessment for Cantonese-speaking People With Aphasia (PWA). The assessment is formulated as a binary classification task to discriminate PWA with high scores of subjective assessment from those with low scores. The 2-layer Gated Recurrent Unit (GRU) and Convolutional Neural Network (CNN) models are applied to realize the end-to-end mapping from basic speech features to the classification outcome. The pathology-specific features used for assessment are learned implicitly by the neural network model. The Class Activation Mapping (CAM) method is utilized to visualize how the learned features contribute to the assessment result. Experimental results show that the end-to-end approach can achieve comparable performance to the conventional two-step approach in the classification task, and the CNN model is able to learn impairment-related features that are similar to the hand-crafted features. The experimental results also indicate that CNN model performs better than 2-layer GRU model in this specific task.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Most research on automatic aphasia assessment [5,9,10,12,13,14,18] has focused on a single aphasia assessment task using automatic speech recognition techniques and a machine learning (ML) framework with a fixed dataset-type.”
From Performance Evaluation of Machine Learning Frameworks for Aphasia Assessment (2021) · cites this paper as [9]
In the citing paper
“The naming, repetition, and auditory verbal comprehension subtests can be automated using customized machine learning [8,11,12].”
From A Comparative Investigation of Automatic Speech Recognition Platforms for Aphasia Assessment Batteries (2023) · cites this paper as [8]
Citations
Cites
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148 · cited 1×
“me acoustic features are extracted and concatenated sequentially as a 2-dimensional time-frequency representation. The CNN model structure described in Table 2 is inspired by the AlexNet [31] and VGG [32] models. It is constructed with a Global Average Pooling (GAP) layer instead of a fully-connected layer after the l…”
MethodImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · cited 1×
“The CNN model structure described in Table 2 is inspired by the AlexNet [31] and VGG [32] models.”
MethodHochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · cited 1×
“al learning rate is set to 10 4 for the sequence-to-one GRU-RNN model and it is set to 10 3 for the CNN model. Model training aims at minimizing the binary cross-entropy loss using the Adam optimizer [34]. Weight decay coecient is set to 5 10 4 for both models to tackle the overtting problem. All models are implemente…”
MethodInfluentialGrad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization
Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.74 · Cited by 24,513
An introduction to ROC analysis
Fawcett, T. (2006) · Pattern Recognition Letters · doi:10.1016/j.patrec.2005.10.010 · Cited by 21,879
(2017) · Cited by 11,625
Learning Deep Features for Discriminative Localization
Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., & Torralba, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.319 · Cited by 11,428 · cited 2×
“. Batch normalization (BN) layer and ReLU activation function are added after each convolutional layer. 4.2 Class Activation Mapping Given a CNN model, the technique of Class Activation Mapping (CAM) [24] is capable of visualizing the class-discriminative regions in the input images. Therefore, the CAM can improve und…”
BackgroundMethodInfluentialOn the Properties of Neural Machine Translation: Encoder–Decoder Approaches
Cho, K., van Merrienboer, B., Bahdanau, D., & Bengio, Y. (2014) · Proceedings of SSST-8, Eighth Workshop on Syntax, Semantics and Structure in Statistical Translation · doi:10.3115/v1/w14-4012 · Cited by 7,099 · cited 1×
“ng numerous neural network structures, Recurrent Neural Network (RNN) has the ability to model sequential signals and to deal with variable-length utterances. LSTM [18] and Gated Recurrent Unit (GRU) [19] are well-established \gating" models that tackle the problem of gradient vanishing in vanilla RNN. The GRU is…”
BackgroundGoogle's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
Wu, Y., Schuster, M., Chen, Z., Le, Q. V., Norouzi, M., Macherey, W., Krikun, M., Cao, Y., Gao, Q., Macherey, K., et al. (2016) · arXiv · doi:10.48550/arxiv.1609.08144 · Cited by 5,319 · cited 1×
“eature extraction and assessment model are optimized separately instead of jointly. Recently, the so-called \end-to-end" approach has demonstrated good successes in ASR [16], machine translation [17], and other applications. It requires relatively little human eort on designing tailored features and shows superio…”
BackgroundSelf-citation
Research works printed but not linked: 1
- Lin, M., Chen, Q., Yan, S. (2013). Network in network. arXiv:cs.NE/1312.4400.
Cited by
Adikari, A., Hernandez, N., Alahakoon, D., Rose, M. L., & Pierce, J. E. (2024) · Disability and Rehabilitation · doi:10.1080/09638288.2023.2199463 · Cited by 54
“The remaining four aimed at providing automated feedback within therapy software, including automated judgements of utterance quality [25] or within immersive VR [30].”
BackgroundMethodAutomatic Assessment of Speech Impairment in Cantonese-Speaking People with Aphasia
Qin, Y., Lee, T., & Kong, A. P. H. (2020) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2019.2956371 · Cited by 32
“In [17], an end-to-end “utteranceto-score” approach was attempted in aphasic speech assessment.”
BackgroundSelf-citationPerformance Evaluation of Machine Learning Frameworks for Aphasia Assessment
Mahmoud, S. S., Kumar, A., Li, Y., Tang, Y., & Fang, Q. (2021) · Sensors · doi:10.3390/s21082582 · Cited by 27
“The CNN models are widely used across various applications including aphasia assessment tasks [9].”
BackgroundMethodInfluentialAI and Aphasia in the Digital Age: A Critical Review
Privitera, A. J., Ng, S. H. S., Kong, A. P.-H., & Weekes, B. S. (2024) · Brain Sciences · doi:10.3390/brainsci14040383 · Cited by 26
“However, two studies by Qin and colleagues utilized automatic speech recognition in their assessments to differentiate between aphasic and non-aphasic speech [72,73].”
BackgroundSelf-citationMahmoud, S. S., Pallaud, R. F., Kumar, A., Faisal, S., Wang, Y., & Fang, Q. (2023) · Sensors · doi:10.3390/s23020857 · Cited by 25
“The naming, repetition, and auditory verbal comprehension subtests can be automated using customized machine learning [8,11,12].”
Influential
Metadata
- Authors
- Qin, Y., Wu, Y., Lee, T., & Kong, A. P. H.
- Affiliations
- Chinese University of Hong Kong (HK)
- University of Central Florida (US)
- Corresponding author
- Qin, Ying
- Venue
- Journal of Signal Processing Systems
- Published
- 2020-02-18
- Volume
- 92
- Issue
- 8
- Pages
- 819-830
- ISSN
- 1939-8018 (ISSN-L)
- 1939-8115
- OpenAlex
- W3006736300
- Semantic Scholar
- 9068f3383f06f4f130f5e8fae99f5861ffd14ca1
- Cited by
- 25
- References
- 40
- Citations by year
- 2026: 1 · 2025: 3 · 2024: 5 · 2023: 9 · 2022: 3 · 2021: 3
- FWCI
- 1.47
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- www.springer.com/tdm (tdm)
- www.springer.com/tdm (vor)
- Locations
- Journal of Signal Processing Systems (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Research Committee of the Chinese University of Hong Kong
- GRF project grant from the Hong Kong Research Grants Council (CUHK14227216)
- National Institutes of Health (NIH-R01-DC010398)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Health Sciences › Health Professions › Speech and Hearing › Voice and Speech Disorders
- Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
- Keywords
- aphasia, Cantonese, end-to-end learning, pathological speech, class activation mapping, gated recurrent unit, convolutional neural networks, representation learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: An End-to-End Approach to Automatic Speech Assessment for Cantonese-speaking People with Aphasia. https://app.cochat.ai/papers/10.1007/s11265-019-01511-3
- The paper
- Qin, Y., Wu, Y., Lee, T., & Kong, A. P. H. (2020). An End-to-End Approach to Automatic Speech Assessment for Cantonese-speaking People with Aphasia. Journal of Signal Processing Systems. https://doi.org/10.1007/s11265-019-01511-3
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.