The DOI matches its Crossref record; 60 of the 60 references it cites were found in a registry or an index.
Deep Learning Enabled Semantic Communications With Speech Recognition and Synthesis
Open accessAbstract
In this paper, we develop a deep learning based semantic communication system for speech transmission, named DeepSC-ST. We take the speech recognition and speech synthesis as the transmission tasks of the communication system, respectively. First, the speech recognition-related semantic features are extracted for transmission by a joint semantic-channel encoder and the text is recovered at the receiver based on the received semantic features, which significantly reduces the required amount of data transmission without performance degradation. Then, we perform speech synthesis at the receiver, which dedicates to re-generate the speech signals by feeding the recognized text and the speaker information into a neural network module. To enable the DeepSC-ST adaptive to dynamic channel environments, we identify a robust model to cope with different channel conditions. According to the simulation results, the proposed DeepSC-ST significantly outperforms conventional communication systems and existing DL-enabled communication systems, especially in the low signal-to-noise ratio (SNR) regime. A software demonstration is further developed as a proof-of-concept of the DeepSC-ST.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A Mathematical Theory of Communication
Shannon, C. E. (1948) · Bell System Technical Journal · doi:10.1002/j.1538-7305.1948.tb00917.x · Cited by 48,949
A tutorial on hidden Markov models and selected applications in speech recognition
Rabiner, L. R. (1989) · Proceedings of the IEEE · doi:10.1109/5.18626 · Cited by 22,743
Hinton, G., Deng, L., Yu, D., Dahl, G., Mohamed, A.-R., Jaitly, N., Senior, A., Vanhoucke, V., Nguyen, P., Sainath, T., et al. (2012) · IEEE Signal Processing Magazine · doi:10.1109/msp.2012.2205597 · Cited by 10,260 · cited 1×
“[25], [26], [27], in which DNN replaces the traditional Gaussian mixture model that is utilized to estimate the distribution of hidden Markov model.”
MethodBidirectional recurrent neural networks
Schuster, M., & Paliwal, K. K. (1997) · IEEE Transactions on Signal Processing · doi:10.1109/78.650093 · Cited by 10,172 · cited 4×
“Then, b is fed into Q BRNN modules, successively, and produces d ∈ RB×GQ×HQ , where the number of GRU units in each BRNN module, Hq , q ∈ [1, 2, . . . , Q], is consistent.”
MethodInfluentialSpeech recognition with deep recurrent neural networks
Graves, A., Mohamed, A.-R., & Hinton, G. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6638947 · Cited by 8,893 · cited 1×
“(LSTM) have been developed [28], [29], which analyses the temporal dependencies of excessively long speech sequences and greatly increases the recognition accuracy compared to the hybrid modeling systems.”
BackgroundA Method for the Construction of Minimum-Redundancy Codes
Huffman, D. (1952) · Proceedings of the IRE · doi:10.1109/jrproc.1952.273898 · Cited by 6,334
Graves, A., Fernández, S., Gomez, F., & Schmidhuber, J. (2006) · Proceedings of the 23rd international conference on Machine learning - ICML '06 · doi:10.1145/1143844.1143891 · Cited by 5,370
WaveNet: a generative model for raw audio
Oord, A. V. D., Dieleman, S., Zen, H., Simonyan, K., Vinyals, O., Graves, A., Kalchbrenner, N., Senior, A., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1609.03499 · Cited by 4,911
Conformer: Convolution-augmented Transformer for Speech Recognition
Gulati, A., Qin, J., Chiu, C.-C., Parmar, N., Zhang, Y., Yu, J., Han, W., Wang, S., Zhang, Z., Wu, Y., et al. (2020) · Interspeech 2020 · doi:10.21437/interspeech.2020-3015 · Cited by 3,150
Conformer: Convolution-augmented Transformer for Speech Recognition
Gulati, A., Qin, J., Chiu, C.-C., Parmar, N., Zhang, Y., Yu, J., Han, W., Wang, S., Zhang, Z., Wu, Y., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.08100 · Cited by 3,150
Long short-term memory recurrent neural network architectures for large scale acoustic modeling
Sak, H., Senior, A., & Beaufays, F. (2014) · Interspeech 2014 · doi:10.21437/interspeech.2014-80 · Cited by 2,973
Natural TTS Synthesis by Conditioning Wavenet on MEL Spectrogram Predictions
Shen, J., Pang, R., Weiss, R. J., Schuster, M., Jaitly, N., Yang, Z., Chen, Z., Zhang, Y., Wang, Y., Skerrv-Ryan, R., et al. (2018) · 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2018.8461368 · Cited by 2,791
The information bottleneck method
Tishby, N., Pereira, F. C., & Bialek, W. (2000) · arXiv · doi:10.48550/arxiv.physics/0004057 · Cited by 2,626 · cited 1×
“information bottleneck [21] framework to identify the optimal tradeoff between compression rate and classification accuracy.”
Backgroundwav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations
Baevski, A., Zhou, H., Mohamed, A., & Auli, M. (2020) · arXiv · doi:10.48550/arxiv.2006.11477 · Cited by 2,369
Deep Speech 2: End-to-End Speech Recognition in English and Mandarin
Amodei, D., Anubhai, R., Battenberg, E., Case, C. T., Casper, J., Catanzaro, B., Chen, J., Chrzanowski, M., Coates, A., Diamos, G., et al. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1512.02595 · Cited by 2,110
Towards End-To-End Speech Recognition with Recurrent Neural Networks
Graves, A., & Jaitly, N. (2014) · Cited by 1,861
Tacotron: towards end-to-end speech synthesis
Wang, Y., Skerry-Ryan, R. J., Stanton, D., Wu, Y., Weiss, R. J., Jaitly, N., Yang, Z., Xiao, Y., Chen, Z., Bengio, S., et al. (2017) · Interspeech 2017 · doi:10.21437/interspeech.2017-1452 · Cited by 1,746 · cited 13×
“2, Tacotron 2 is adopted to reconstruct the speech sample sequences, M̂ , as close to the input sample sequences, M , as possible.”
MethodInfluentialAcoustic Modeling Using Deep Belief Networks
Mohamed, A.-R., Dahl, G. E., & Hinton, G. (2012) · IEEE Transactions on Audio, Speech, and Language Processing · doi:10.1109/tasl.2011.2109382 · Cited by 1,727
Deep Learning Enabled Semantic Communication Systems
Xie, H., Qin, Z., Li, G. Y., & Juang, B.-H. (2021) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2021.3071210 · Cited by 1,602
Self-citationPitch-synchronous waveform processing techniques for text-to-speech synthesis using diphones
Moulines, E., & Charpentier, F. (1990) · Speech Communication · doi:10.1016/0167-6393(90)90021-z · Cited by 1,244
Unit selection in a concatenative speech synthesis system using a large speech database
Hunt, A. J., & Black, A. W. (2002) · 1996 IEEE International Conference on Acoustics, Speech, and Signal Processing Conference Proceedings · doi:10.1109/icassp.1996.541110 · Cited by 1,190
Statistical parametric speech synthesis using deep neural networks
Zen, H., Senior, A., & Schuster, M. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6639215 · Cited by 809
Deep Learning in Physical Layer Communications
Qin, Z., Ye, H., Li, G. Y., & Juang, B.-H. F. (2019) · IEEE Wireless Communications · doi:10.1109/mwc.2019.1800601 · Cited by 692
Self-citationSemantic Communication Systems for Speech Transmission
Weng, Z., & Qin, Z. (2021) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2021.3087240 · Cited by 663
Self-citationLLR-Based Successive Cancellation List Decoding of Polar Codes
Balatsoukas-Stimming, A., Bastani Parizi, M., & Burg, A. (2015) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2015.2439211 · Cited by 631
Streaming End-to-end Speech Recognition for Mobile Devices
He, Y., Sainath, T. N., Prabhavalkar, R., McGraw, I., Alvarez, R., Zhao, D., Rybach, D., Kannan, A., Wu, Y., Pang, R., et al. (2019) · ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2019.8682336 · Cited by 601
FastSpeech 2: Fast and High-Quality End-to-End Text to Speech
Ren, Y., Hu, C., Tan, X., Qin, T., Zhao, S., Zhao, Z., & Liu, T.-Y. (2020) · arXiv · doi:10.48550/arxiv.2006.04558 · Cited by 580
Task-Oriented Multi-User Semantic Communications
Xie, H., Qin, Z., Tao, X., & Letaief, K. B. (2022) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2022.3191326 · Cited by 458
Self-citationDeep Learning for Joint Source-Channel Coding of Text
Farsad, N., Rao, M., & Goldsmith, A. (2018) · 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2018.8461983 · Cited by 438
Zhang, Q., Lu, H., Sak, H., Tripathi, A., McDermott, E., Koo, S., & Kumar, S. (2020) · ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp40776.2020.9053896 · Cited by 433
IEEE Standard for Floating-Point Arithmetic
(2019) · doi:10.1109/ieeestd.2019.8766229 · Cited by 341
Learning Task-Oriented Communication for Edge Inference: An Information Bottleneck Approach
Shao, J., Mao, Y., & Zhang, J. (2022) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2021.3126087 · Cited by 338
Toward Semantic Communications: Deep Learning-Based Image Semantic Coding
Huang, D., Gao, F., Tao, X., Du, Q., & Lu, J. (2023) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2022.3221999 · Cited by 331
Self-citationThe adaptive multirate wideband speech codec (AMR-WB)
Bessette, B., Salami, R., Lefebvre, R., Jelinek, M., Rotola-Pukkila, J., Vainio, J., Mikkola, H., & Jarvinen, K. (2002) · IEEE Transactions on Speech and Audio Processing · doi:10.1109/tsa.2002.804299 · Cited by 315
(2017) · International Conference on Learning Representations · Cited by 314
Deep voice 3: scaling text-to-speech with convolutional sequence learning
Ping, W., Peng, K., Gibiansky, A., Arik, S. O., Kannan, A., Narang, S., Raiman, J., & Miller, J. (2017) · arXiv · doi:10.48550/arxiv.1710.07654 · Cited by 313
Highway long short-term memory RNNS for distant speech recognition
Zhang, Y., Chen, G., Yu, D., Yao, K., Khudanpur, S., & Glass, J. (2016) · 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2016.7472780 · Cited by 290
Speaker-Dependent WaveNet Vocoder
Tamamori, A., Hayashi, T., Kobayashi, K., Takeda, K., & Toda, T. (2017) · Interspeech 2017 · doi:10.21437/interspeech.2017-314 · Cited by 269
Zen, H., & Sak, H. (2015) · 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2015.7178816 · Cited by 262
Wireless Semantic Communications for Video Conferencing
Jiang, P., Wen, C.-K., Jin, S., & Li, G. Y. (2023) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2022.3221968 · Cited by 255
Self-citationSemantic Communications: Principles and Challenges
Qin, Z., Tao, X., Lü, J., Tong, W., & Li, G. Y. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2201.01389 · Cited by 254
Self-citationWu, Z., Valentini-Botinhao, C., Watts, O., & King, S. (2015) · 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2015.7178814 · Cited by 238
High Fidelity Speech Synthesis with Adversarial NetworksPreprint
Bińkowski, M., Donahue, J., Dieleman, S., Clark, A., Elsen, E., Casagrande, N., Cobo, L. C., & Simonyan, K. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.11646 · Cited by 233
Exploring neural transducers for end-to-end speech recognition
Battenberg, E., Chen, J., Child, R., Coates, A., Li, Y. G. Y., Liu, H., Satheesh, S., Sriram, A., & Zhu, Z. (2017) · 2017 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) · doi:10.1109/asru.2017.8268937 · Cited by 227
Deep Source-Channel Coding for Sentence Semantic Transmission With HARQ
Jiang, P., Wen, C.-K., Jin, S., & Li, G. Y. (2022) · IEEE Transactions on Communications · doi:10.1109/tcomm.2022.3180997 · Cited by 199
Self-citationHubert: How Much Can a Bad Teacher Benefit ASR Pre-Training?
Hsu, W.-N., Tsai, Y.-H. H., Bolte, B., Salakhutdinov, R., & Mohamed, A. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9414460 · Cited by 195
Improving RNN Transducer Modeling for End-to-End Speech Recognition
Li, J., Zhao, R., Hu, H., & Gong, Y. (2019) · 2019 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) · doi:10.1109/asru46091.2019.9003906 · Cited by 170
Nine Challenges in Artificial Intelligence and Wireless Communications for 6G
Tong, W., & Li, G. Y. (2022) · IEEE Wireless Communications · doi:10.1109/mwc.006.2100543 · Cited by 166
Self-citationTask-Oriented Image Transmission for Scene Classification in Unmanned Aerial Systems
Kang, X., Song, B., Guo, J., Qin, Z., & Yu, F. R. (2022) · IEEE Transactions on Communications · doi:10.1109/tcomm.2022.3182325 · Cited by 158
Self-citationGenerative Pre-Training for Speech with Autoregressive Predictive Coding
Chung, Y.-A., & Glass, J. (2020) · ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp40776.2020.9054438 · Cited by 157
Deep and Wide: Multiple Layers in Automatic Speech Recognition
Morgan, N. H. (2011) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2011.2116010 · Cited by 149
Flow-TTS: A Non-Autoregressive Network for Text to Speech Based on Flow
Miao, C., Liang, S., Chen, M., Ma, J., Wang, S., & Xiao, J. Z. (2020) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp40776.2020.9054484 · Cited by 139
Robust Semantic Communications Against Semantic Noise
Hu, Q., Zhang, G., Qin, Z., Cai, Y., Yu, G., & Li, G. Y. (2022) · IEEE Vehicular Technology Conference · doi:10.1109/vtc2022-fall57202.2022.10012843 · Cited by 137
Self-citationA new communication paradigm: from bit accuracy to semantic fidelityPreprint
Shi, G., Gao, D., Song, X., Chai, J., Yang, M., Xie, X., Li, L., & Li, X. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.12649 · Cited by 67
Learning Semantics: An Opportunity for Effective 6G Communications
Sana, M., & Strinati, E. C. (2022) · IEEE Consumer Communications and Networking Conference · doi:10.1109/ccnc49033.2022.9700645 · Cited by 67
Preserving quality of information by using semantic relationships
Basu, P., Bao, J., Dean, M., & Hendler, J. (2013) · Pervasive and Mobile Computing · doi:10.1016/j.pmcj.2013.07.013 · Cited by 57
Federated Learning for Audio Semantic Communication
Tong, H., Yang, Z., Wang, S., Hu, Y., Semiari, O., Saad, W., & Yin, C. (2021) · Frontiers in Communications and Networks · doi:10.3389/frcmn.2021.734402 · Cited by 55
Semantic Communications for Speech Recognition
Weng, Z., Qin, Z., & Li, G. Y. (2021) · Globecom · doi:10.1109/globecom46510.2021.9685250 · Cited by 55
Self-citationLife-long Learning for Reasoning-based Semantic Communication
Liang, J., Xiao, Y. L., Li, Y., Shi, G., & Bennis, M. (2022) · IEEE International Conference on Communications workshops · doi:10.1109/iccworkshops53468.2022.9814575 · Cited by 38
Wireless Image Retrieval at the Edge
Jankowski, M., Gunduz, D., & Mikolajczyk, K. (2021) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2020.3036955 · Cited by 3 · cited 1×
“has been investigated in [19], which imposes IoT devices to compress images to the low computational complexity and to reduce the required transmission bandwidth.”
Background
Cited by
Semantic Communication: A Survey on Research Landscape, Challenges, and Future Directions
Getu, T. M., Kaddoum, G., & Bennis, M. (2024) · Proceedings of the IEEE · doi:10.1109/jproc.2024.3520707 · Cited by 110
Metadata
- Authors
- Weng, Z., Qin, Z., Tao, X., Pan, C., Liu, G., & Li, G. Y.
- Venue
- IEEE Transactions on Wireless Communications
- Published
- 2023-09
- Volume
- 22
- Issue
- 9
- Pages
- 6227-6240
- ISSN
- 1536-1276 (ISSN-L)
- 1558-2248
- OpenAlex
- W4319302797
- Cited by
- 286
- References
- 60
- FWCI
- 20.82
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.