Speech Technology for Healthcare: Opportunities, Challenges, and State of the Art
The DOI matches its Crossref record; 39 of the 39 references it cites were found in a registry or an index.
Abstract
Speech technology is not appropriately explored even though modern advances in speech technology-especially those driven by deep learning (DL) technology-offer unprecedented opportunities for transforming the healthcare industry. In this paper, we have focused on the enormous potential of speech technology for revolutionising the healthcare domain. More specifically, we review the state-of-the-art approaches in automatic speech recognition (ASR), speech synthesis or text to speech (TTS), and health detection and monitoring using speech signals. We also present a comprehensive overview of various challenges hindering the growth of speech-based services in healthcare. To make speech-based healthcare solutions more prevalent, we discuss open issues and suggest some possible research directions aimed at fully leveraging the advantages of other technologies for making speech-based healthcare solutions more effective.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Representation Learning: A Review and New Perspectives
Bengio, Y., Courville, A., & Vincent, P. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.50 · Cited by 13,001
Explaining and Harnessing Adversarial Examples
Goodfellow, I. J., Shlens, J., & Szegedy, C. (2014) · arXiv · doi:10.48550/arxiv.1412.6572 · Cited by 10,381
Bidirectional recurrent neural networks
Schuster, M., & Paliwal, K. K. (1997) · IEEE Transactions on Signal Processing · doi:10.1109/78.650093 · Cited by 10,172 · cited 9×
“Recurrent neural networks (RNNs) define a special DL architecture that uses recurrent connections within layers with the capability of processing previously processed inputs.”
InfluentialAn argument for basic emotions
Ekman, P. (1992) · Cognition and Emotion · doi:10.1080/02699939208411068 · Cited by 9,646 · cited 1×
“Humans feel a vast range of comfortable and uncomfortable emotions [74] and it has been argued that emotional discomfort is a universal human experience [75].”
BackgroundTowards Evaluating the Robustness of Neural Networks
Carlini, N., & Wagner, D. (2017) · 2017 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp.2017.49 · Cited by 8,859
Extracting and composing robust features with denoising autoencoders
Vincent, P., Larochelle, H., Bengio, Y., & Manzagol, P.-A. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390294 · Cited by 7,436
“Librispeech: an ASR corpus based on public domain audio books,”
Panayotov, V., Chen, G., Povey, D., & Khudanpur, S. (2015) · 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2015.7178964 · Cited by 6,072
DeepFool: A Simple and Accurate Method to Fool Deep Neural Networks
Moosavi-Dezfooli, S.-M., Fawzi, A., & Frossard, P. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.282 · Cited by 5,317 · cited 2×
“tack (JSMA) [194], DeepFool [195], and Carlini and Wagner attacks [196] follow this method.”
MethodWaveNet: a generative model for raw audio
Oord, A. V. D., Dieleman, S., Zen, H., Simonyan, K., Vinyals, O., Graves, A., Kalchbrenner, N., Senior, A., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1609.03499 · Cited by 4,911
The Limitations of Deep Learning in Adversarial Settings
Papernot, N., McDaniel, P., Jha, S., Fredrikson, M., Celik, Z. B., & Swami, A. (2016) · 2016 IEEE European Symposium on Security and Privacy (EuroS&P) · doi:10.1109/eurosp.2016.36 · Cited by 3,993
SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition
Park, D. S., Chan, W., Zhang, Y., Chiu, C.-C., Zoph, B., Cubuk, E. D., & Le, Q. V. (2019) · Interspeech 2019 · doi:10.21437/interspeech.2019-2680 · Cited by 3,558
Privacy-preserving deep learning
Shokri, R., & Shmatikov, V. (2015) · Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/2810103.2813687 · Cited by 2,356 · cited 1×
“Similarly, federated learning [213] is another alternative solution to preserve users’ privacy.”
BackgroundListen, attend and spell: A neural network for large vocabulary conversational speech recognition
Chan, W., Jaitly, N., Le, Q., & Vinyals, O. (2016) · 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2016.7472621 · Cited by 2,226 · cited 1×
“Various sequence-to-sequence models including Recurrent Neural Network Transducer (RNNT) [93], Neural Transducer [94], Listen, Attend and Spell (LAS) [95], Recurrent Neural Aligner (RNA)[96],andMonotonicAlignments[97]havebeenexplored intheliterature.”
MethodUniversals and cultural differences in the judgments of facial expressions of emotion.
Ekman, P., Friesen, W. V., O'Sullivan, M., Chan, A., Diacoyanni-Tarlatzis, I., Heider, K., Krause, R., LeCompte, W. A., Pitcairn, T., Ricci-Bitti, P. E., et al. (1987) · Journal of Personality and Social Psychology · doi:10.1037//0022-3514.53.4.712 · Cited by 1,763
Tacotron: towards end-to-end speech synthesis
Wang, Y., Skerry-Ryan, R. J., Stanton, D., Wu, Y., Weiss, R. J., Jaitly, N., Yang, Z., Xiao, Y., Chen, Z., Bengio, S., et al. (2017) · Interspeech 2017 · doi:10.21437/interspeech.2017-1452 · Cited by 1,746
Conversational agents in healthcare: a systematic review
Laranjo, L., Dunn, A. G., Tong, H. L., Kocaballi, A. B., Chen, J., Bashir, R., Surian, D., Gallego, B., Magrabi, F., Lau, A. Y. S., et al. (2018) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocy072 · Cited by 1,388
Adversarial attacks on medical machine learning
Finlayson, S. G., Bowers, J. D., Ito, J., Zittrain, J. L., Beam, A. L., & Kohane, I. S. (2019) · Science · doi:10.1126/science.aaw4399 · Cited by 1,088 · cited 1×
“Some other popular adversarial attacks against speech-based systems include [200]–[204].”
BackgroundState-of-the-Art Speech Recognition with Sequence-to-Sequence Models
Chiu, C.-C., Sainath, T. N., Wu, Y., Prabhavalkar, R., Nguyen, P., Chen, Z., Kannan, A., Weiss, R. J., Rao, K., Gonina, E., et al. (2018) · 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2018.8462105 · Cited by 1,043
“Waveglow: a flow-based generative network for speech synthesis,”
Prenger, R., Valle, R., & Catanzaro, B. (2019) · ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2019.8683143 · Cited by 1,004
Efficient Neural Audio Synthesis
Kalchbrenner, N., Elsen, E., Simonyan, K., Noury, S., Casagrande, N., Lockhart, E., Stimberg, F., Oord, A. V. D., Dieleman, S., & Kavukcuoglu, K. (2018) · arXiv · doi:10.48550/arxiv.1802.08435 · Cited by 961
Linguistic Features Identify Alzheimer’s Disease in Narrative Speech
Fraser, K. C., Meltzer, J. A., & Rudzicz, F. (2015) · Journal of Alzheimer's Disease · doi:10.3233/jad-150520 · Cited by 797 · cited 1×
“In [128], Fraser et al. explored the use of linguistic features for the identification of Alzheimer’s.”
BackgroundNeural Speech Synthesis with Transformer Network
Li, N., Liu, S., Liu, Y., Zhao, S., & Liu, M. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33016706 · Cited by 754
MelGAN: generative adversarial networks for conditional waveform synthesis
Kumar, K., Kumar, R., de Boissiere, T., Gestin, L., Teoh, W. Z., Sotelo, J., de Brebisson, A., Bengio, Y., & Courville, A. (2019) · arXiv · doi:10.48550/arxiv.1910.06711 · Cited by 722
The Microsoft 2017 Conversational Speech Recognition System
Xiong, W., Wu, L., Alleva, F., Droppo, J., Huang, X., & Stolcke, A. (2018) · 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2018.8461870 · Cited by 472 · cited 1×
“have become the state-of-the-art by showing huge improvements compared to previous conventional system [90].”
ResultAutomatic speech analysis for the assessment of patients with predementia and Alzheimer's disease
König, A., Satt, A., Sorin, A., Hoory, R., Toledo‐Ronen, O., Derreumaux, A., Manera, V., Verhey, F., Aalten, P., Robert, P. H., et al. (2015) · Alzheimer's & Dementia: Diagnosis, Assessment & Disease Monitoring · doi:10.1016/j.dadm.2014.11.012 · Cited by 380 · cited 1×
“Prosody refers to melody and rhythm of speech and prosodic features include the feature related to the length, tone, accent, stress, intonation, and few others [25], [26].”
BackgroundSpoken Language Derived Measures for Detecting Mild Cognitive Impairment
Roark, B., Mitchell, M., Hosom, J.-P., Hollingshead, K., & Kaye, J. (2011) · IEEE Transactions on Audio, Speech, and Language Processing · doi:10.1109/tasl.2011.2112351 · Cited by 355
Carlson, L. E., & Bultz, B. D. (2003) · Journal of Psychosomatic Research · doi:10.1016/s0022-3999(03)00514-2 · Cited by 351 · cited 1×
“Distress is major emotional suffering and highly prevalent in patients with a chronic disease like cancer [77].”
BackgroundThe impact of psychological interventions on medical cost offset: A meta-analytic review.
Chiles, J. A., Lambert, M. J., & Hatch, A. L. (1999) · Clinical Psychology: Science and Practice · doi:10.1093/clipsy.6.2.204 · Cited by 301
Detection of Pathological Voice Using Cepstrum Vectors: A Deep Learning Approach
Fang, S.-H., Tsao, Y., Hsiao, M.-J., Chen, J.-Y., Lai, Y.-H., Lin, F.-C., & Wang, C.-T. (2019) · Journal of Voice · doi:10.1016/j.jvoice.2018.02.003 · Cited by 278
Emotion-aware connected healthcare big data towards 5G
Hossain, M. S., & Muhammad, G. (2018) · IEEE Internet of Things Journal · doi:10.1109/jiot.2017.2772959 · Cited by 273 · cited 1×
“G enabled emotion-aware healthcare framework was proposed in [145].”
BackgroundQayyum, A., Usama, M., Qadir, J., & Al-Fuqaha, A. (2020) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2020.2975048 · Cited by 257
Self-citationRWTH ASR Systems for LibriSpeech: Hybrid vs Attention
Lüscher, C., Beck, E., Irie, K., Kitza, M., Michel, W., Zeyer, A., Schlüter, R., & Ney, H. (2019) · Interspeech 2019 · doi:10.21437/interspeech.2019-1780 · Cited by 241
How 5g wireless (and concomitant technologies) will revolutionize healthcare?
Latif, S., Qadir, J., Farooq, S., & Imran, M. (2017) · Future Internet · doi:10.3390/fi9040093 · Cited by 219
Self-citationA systematic review of speech recognition technology in health care
Johnson, M., Lapkin, S., Long, V., Sanchez, P., Suominen, H., Basilakis, J., & Dawson, L. (2014) · BMC Medical Informatics and Decision Making · doi:10.1186/1472-6947-14-94 · Cited by 141 · cited 1×
“This work builds upon previous papers with a limited focus that aimed to demonstrate the potential of speech recognition for healthcare [8]–[13] or discuss the use of DL for health-related sub-challenges on publicly available datasets [14].”
ResultCoiera, E., Kocaballi, B., Halamka, J., & Laranjo, L. (2018) · npj Digital Medicine · doi:10.1038/s41746-018-0066-9 · Cited by 132
Lopez-de-Ipina, K., Martinez-de-Lizarduy, U., Calvo, P. M., Mekyska, J., Beitia, B., Barroso, N., Estanga, A., Tainta, M., & Ecay-Torres, M. (2018) · Current Alzheimer Research · doi:10.2174/1567205014666171120143800 · Cited by 65
Efficiency and safety of speech recognition for documentation in the electronic health record
Hodgson, T., Magrabi, F., & Coiera, E. (2017) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocx073 · Cited by 63 · cited 1×
“Furthermore, improved system interoperability and workflow are needed for their successful integration in the clinical setting [119].”
Background
Cited by
A review of deep learning techniques for speech processing
Mehrish, A., Majumder, N., Bharadwaj, R., Mihalcea, R., & Poria, S. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.101869 · Cited by 389
Facilitating retail customers’ use of AI-based virtual assistants: A meta-analysis
Blut, M., Wünderlich, N. V., & Brock, C. (2024) · Journal of Retailing · doi:10.1016/j.jretai.2024.04.001 · Cited by 75
Zhang, J., Wu, J., Qiu, Y., Song, A., Li, W., Li, X., & Liu, Y. (2023) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2022.106517 · Cited by 69
“In addition, intelligent speech technology (IST) plays a critical role in smart hospitals because language is the most natural mean of communication between doctors and patients and contains much information, such as patients ’ identity, age, emotion, and even symptoms of diseases [10].”
Sparks of Large Audio Models: A Survey and Outlook
Latif, S., Shoukat, M., Shamshad, F., Usama, M., Ren, Y., Cuayáhuitl, H., Wang, W., Zhang, X., Togneri, R., Cambria, E., et al. (2023) · arXiv · doi:10.48550/arxiv.2308.12792 · Cited by 17
“These applications range from voice-activated assistants like Siri and Alexa [1, 2] to transcription services [3], and extend to telecommunication systems [4] and hearing aids [5].”
BackgroundSelf-citation
Metadata
- Authors
- Latif, S., Qadir, J., Qayyum, A., Usama, M., & Younis, S.
- Venue
- IEEE Reviews in Biomedical Engineering
- Published
- 2021
- Volume
- 14
- Pages
- 342-356
- ISSN
- 1937-3333 (ISSN-L)
- 1941-1189
- OpenAlex
- W3038342317
- Cited by
- 173
- References
- 39
- FWCI
- 6.82
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.