Semantic-Preserved Communication System for Highly Efficient Speech Transmission
Open accessThe DOI matches its Crossref record; 9 of the 9 references it cites were found in a registry or an index.
Abstract
Deep learning (DL) based semantic communication methods have been explored for the efficient transmission of images, text, and speech in recent years. In contrast to traditional wireless communication methods that focus on the transmission of abstract symbols, semantic communication approaches attempt to achieve better transmission efficiency by only sending the semantic-related information of the source data. In this paper, we consider semantic-oriented speech transmission which transmits only the semantic-relevant information over the channel for the speech recognition task, and a compact additional set of semantic-irrelevant information for the speech reconstruction task. We propose a novel end-to-end DL-based transceiver which extracts and encodes the semantic information from the input speech spectrums at the transmitter and outputs the corresponding transcriptions from the decoded semantic information at the receiver. In particular, we employ a soft alignment module and a redundancy removal module to extract only the text-related semantic features while dropping semantically redundant content, greatly reducing the amount of semantic redundancy compared to existing methods. We also propose a semantic correction module to further correct the predicted transcription with semantic knowledge by leveraging a pretrained language model. For the speech to speech transmission, we further include a CTC alignment module that extracts a small number of additional semantic-irrelevant but speech-related information, such as duration, pitch, power and speaker identification of the speech for the better reconstruction of the original speech signals at the receiver. We also introduce a two-stage training scheme which speeds up the training of the proposed DL model. The simulation results confirm that our proposed method outperforms current methods in terms of the accuracy of the predicted text for the speech to text transmission and the quality of the recovered speech signals for the speech to speech transmission, and significantly improves transmission efficiency. More specifically, the proposed method only sends 16% of the amount of the transmitted symbols required by the existing methods while achieving about a 10% reduction in WER for the speech to text transmission. For the speech to speech transmission, it results in an even more remarkable improvement in terms of transmission efficiency with only 0.2% of the amount of the transmitted symbols required by the existing method while preserving the comparable quality of the reconstructed speech signals.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
“Librispeech: an ASR corpus based on public domain audio books,”
Panayotov, V., Chen, G., Povey, D., & Khudanpur, S. (2015) · 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2015.7178964 · Cited by 6,072
Graves, A., Fernández, S., Gomez, F., & Schmidhuber, J. (2006) · Proceedings of the 23rd international conference on Machine learning - ICML '06 · doi:10.1145/1143844.1143891 · Cited by 5,370
A Learning Algorithm for Continually Running Fully Recurrent Neural Networks
Williams, R. J., & Zipser, D. (1989) · Neural Computation · doi:10.1162/neco.1989.1.2.270 · Cited by 4,369 · cited 1×
“…S ; and L ce denotes the cross-entropy loss between the predicted transcription (cid:98) G , and the ground truth for the transcription G , which is given by Due to the presence of a feedback loop in the soft alignment module, we use the teacher forcing strategy [39] to achieve fast convergence.”
Deep Learning Enabled Semantic Communication Systems
Xie, H., Qin, Z., Li, G. Y., & Juang, B.-H. (2021) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2021.3071210 · Cited by 1,602
Neural Speech Synthesis with Transformer Network
Li, N., Liu, S., Liu, Y., Zhao, S., & Liu, M. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33016706 · Cited by 754 · cited 2×
“sive text to speech method [38] while the other parts remain the same, referred to as Proposed + AR.”
FastSpeech 2: Fast and High-Quality End-to-End Text to Speech
Ren, Y., Hu, C., Tan, X., Qin, T., Zhao, S., Zhao, Z., & Liu, T.-Y. (2020) · arXiv · doi:10.48550/arxiv.2006.04558 · Cited by 580
Mel-cepstral distance measure for objective speech quality assessment
Kubichek, R. (2002) · Proceedings of IEEE Pacific Rim Conference on Communications Computers and Signal Processing · doi:10.1109/pacrim.1993.407206 · Cited by 534 · cited 8×
“The smaller the MCD is, the closer the reconstructed speech spectrum is to the original spectrum.”
InfluentialA Lite Distributed Semantic Communication System for Internet of Things
Xie, H., & Qin, Z. (2021) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2020.3036968 · Cited by 497 · cited 1×
“The authors in [2] have also proposed an efficient and robust semantic-oriented transmission scheme, the deep learning model of which was then further compressed to be able to work on IoT devices [13] using the model quantization and pruning.”
Cited by
Semantic Communication: A Survey on Research Landscape, Challenges, and Future Directions
Getu, T. M., Kaddoum, G., & Bennis, M. (2024) · Proceedings of the IEEE · doi:10.1109/jproc.2024.3520707 · Cited by 110
A novel semantic-based multi-packet parallel transmission scheme for networked control systems
Yang, H., Peng, C., Cao, Z., & Zhang, X.-M. (2025) · Automatica · doi:10.1016/j.automatica.2025.112120 · Cited by 44
Metadata
- Authors
- Han, T., Yang, Q., Shi, Z., He, S., & Zhang, Z.
- Venue
- IEEE Journal on Selected Areas in Communications
- Published
- 2023-01
- Volume
- 41
- Issue
- 1
- Pages
- 245-259
- ISSN
- 0733-8716 (ISSN-L)
- 1558-0008
- OpenAlex
- W4313350185
- Cited by
- 217
- References
- 9
- FWCI
- 13.73
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.