Automatic Speech Assessment for People with Aphasia Using TDNN-BLSTM with Multi-Task Learning
Open accessThe DOI matches its Crossref record; 26 of the 26 references it cites were found in a registry or an index.
Abstract
This paper describes an investigation on automatic speech assessment for people with aphasia (PWA) using a DNN based automatic speech recognition (ASR) system. The main problems being addressed are the lack of training speech in the intended application domain and the relevant degradation of ASR performance for impaired speech of PWA. We adopt the TDNN-BLSTM structure for acoustic modeling and apply the technique of multi-task learning with large amount of domain-mismatched data. This leads to a significant improvement on the recognition accuracy, as compared with a conventional single-task learning DNN system. To facilitate the extraction of robust text features for quantifying language impairment in PWA speech, we propose to incorporate N-best hypotheses and confusion network representation of the ASR output. The severity of impairment is predicted from text features and supra-segmental duration features using different regression models. Experimental results show a high correlation of 0.842 between the predicted severity level and the subjective Aphasia Quotient score.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Regression shrinkage and selection via the lasso
Tibshirani, R. (1996) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1996.tb02080.x · Cited by 51,482 · cited 1×
“In order to select the most effective features and reduce the feature di-mension for the subsequent regression process, we jointly consider the ranking order of 13 candidate features suggested by the LASSO regression [34, 35] and the correlation between each feature and the AQ value.”
MethodEfficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568 · cited 1×
“The syllable vectors were obtained from a continuous bag-of-words model [31] trained with syllable-level transcriptions of the training set of CanAB.”
MethodBatch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S. (2024) · arXiv (Cornell University) · doi:10.57702/o9raffed · Cited by 14,214
Long short-term memory recurrent neural network architectures for large scale acoustic modeling
Sak, H., Senior, A., & Beaufays, F. (2014) · Interspeech 2014 · doi:10.21437/interspeech.2014-80 · Cited by 2,973 · cited 1×
“Time delay neural network (TDNN) [11] and long-short-term memory recurrent neural network (LSTM-RNN) [12] are well known of the capability of capturing long-term temporal dependency of acoustic events.”
BackgroundHybrid speech recognition with Deep Bidirectional LSTM
Graves, A., Jaitly, N., & Mohamed, A.-R. (2013) · 2013 IEEE Workshop on Automatic Speech Recognition and Understanding · doi:10.1109/asru.2013.6707742 · Cited by 1,736 · cited 1×
“Bidirectional LSTM (BLSTM) [13] is an extension to LSTM with both preceding and succeeding contexts considered.”
BackgroundA time delay neural network architecture for efficient modeling of long temporal contexts
Peddinti, V., Povey, D., & Khudanpur, S. (2015) · Interspeech 2015 · doi:10.21437/interspeech.2015-647 · Cited by 986 · cited 1×
“Time delay neural network (TDNN) [11] and long-short-term memory recurrent neural network (LSTM-RNN) [12] are well known of the capability of capturing long-term temporal dependency of acoustic events.”
BackgroundA pitch extraction algorithm tuned for automatic speech recognition
Ghahremani, P., BabaAli, B., Povey, D., Riedhammer, K., Trmal, J., & Khudanpur, S. (2014) · 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2014.6854049 · Cited by 288 · cited 2×
“Pitch features have been shown useful to boost ASR performance on tonal language like Cantonese [29].”
BackgroundAutomated classification of primary progressive aphasia subtypes from narrative speech transcripts
Fraser, K. C., Meltzer, J. A., Graham, N. L., Leonard, C., Hirst, G., Black, S. E., & Rochon, E. (2014) · Cortex · doi:10.1016/j.cortex.2012.12.006 · Cited by 229 · cited 1×
“In [2, 3], Fraser et al. used acoustic and text features for automatic classification of sub-types of primary progressive aphasia.”
BackgroundFinding consensus among words: lattice-based word error minimization
Mangu, L., Brill, E., & Stolcke, A. (1999) · 6th European Conference on Speech Communication and Technology · doi:10.21437/eurospeech.1999-127 · Cited by 214 · cited 1×
“On the other hand, rich representation of ASR output, such as N-best hypotheses [15] and confusion networks [16], has been widely used in spoken language understanding and speech translation [17, 18].”
BackgroundExplicit word error minimization in n-best list rescoring
Stolcke, A., Konig, Y., & Weintraub, M. (1997) · European Conference on Speech Communication and Technology (Eurospeech) · doi:10.21437/eurospeech.1997-68 · Cited by 153 · cited 1×
“On the other hand, rich representation of ASR output, such as N-best hypotheses [15] and confusion networks [16], has been widely used in spoken language understanding and speech translation [17, 18].”
Background
Cited by
Careless Whisper: Speech-to-Text Hallucination Harms
Koenecke, A., Choi, A. S. G., Mei, K. X., Schellmann, H., & Sloane, M. (2024) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3630106.3658996 · Cited by 219
“Research shows that patients exhibiting aphasia are usually older than stroke patients who do not exhibit aphasia, while those stroke patients who are younger are more likely to exhibit less fluent (e.g., non-fluent or Broca’s type) aphasia [7].”
BackgroundAdikari, A., Hernandez, N., Alahakoon, D., Rose, M. L., & Pierce, J. E. (2024) · Disability and Rehabilitation · doi:10.1080/09638288.2023.2199463 · Cited by 54
An Efficient Deep Learning Based Method for Speech Assessment of Mandarin-Speaking Aphasic Patients
Mahmoud, S. S., Kumar, A., Tang, Y., Li, Y., Gu, X., Fu, J., & Fang, Q. (2020) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2020.3011104 · Cited by 36
“In[22],theautomaticspeechassessment forPWAreliesonthemanualtranscriptionofaudiospeechinto Chinesecharacters.”
Automatic Assessment of Speech Impairment in Cantonese-Speaking People with Aphasia
Qin, Y., Lee, T., & Kong, A. P. H. (2020) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2019.2956371 · Cited by 32
“In the previous study [25], we incorporated rich representation of ASR output (i.”
MethodSelf-citationPerformance Evaluation of Machine Learning Frameworks for Aphasia Assessment
Mahmoud, S. S., Kumar, A., Li, Y., Tang, Y., & Fang, Q. (2021) · Sensors · doi:10.3390/s21082582 · Cited by 27
Metadata
- Authors
- Qin, Y., Lee, T., Feng, S., & Kong, A. P. H.
- Venue
- Interspeech 2018
- Publisher
- ISCA
- Published
- 2018-09-02
- Pages
- 3418-3422
- ISSN
- 2308-457X (ISSN-L)
- OpenAlex
- W2889294685
- Semantic Scholar
- 27b92a0da0ee4014c16770499764f9b06386050f
- Cited by
- 30
- References
- 26
- Citations by year
- 2025: 3 · 2024: 2 · 2023: 7 · 2022: 3 · 2021: 6 · 2020: 3 · 2019: 5 · 2018: 1
- FWCI
- 1.52
- Type
- conference paper
- Language
- en
- Open access
- Locations
- Interspeech (USB) (publishedVersion)
- Journal of International Crisis and Risk Communication Research (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Health Sciences › Health Professions › Speech and Hearing › Voice and Speech Disorders
- Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
- Keywords
- aphasia, multi-task learning, acoustic modeling, confusion networks, aphasia quotient
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Automatic Speech Assessment for People with Aphasia Using TDNN-BLSTM with Multi-Task Learning. https://app.cochat.ai/papers/10.21437/interspeech.2018-1630
- The paper
- Qin, Y., Lee, T., Feng, S., & Kong, A. P. H. (2018). Automatic Speech Assessment for People with Aphasia Using TDNN-BLSTM with Multi-Task Learning. Interspeech 2018. https://doi.org/10.21437/interspeech.2018-1630
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.