The DOI matches its Crossref record; 48 of the 48 references it cites were found in a registry or an index.
Task-Oriented Multi-User Semantic Communications
Open accessAbstract
While semantic communications have shown the potential in the case of single-modal single-users, its applications to the multi-user scenario remain limited. In this paper, we investigate deep learning (DL) based multi-user semantic communication systems for transmitting single-modal data and multimodal data, respectively. We adopt three intelligent tasks, including, image retrieval, machine translation, and visual question answering (VQA) as the transmission goal of semantic communication systems. We propose a Transformer based framework to unify the structure of transmitters for different tasks. For the single-modal multi-user system, we propose two Transformer based models, named, DeepSC-IR and DeepSC-MT, to perform image retrieval and machine translation, respectively. In this case, DeepSC-IR is trained to optimize the distance in embedding space between images and DeepSC-MT is trained to minimize the semantic errors by recovering the semantic meaning of sentences. For the multimodal multi-user system, we develop a Transformer enabled model, named, DeepSC-VQA, for the VQA task by extracting text-image information at the transmitters and fusing it at the receiver. In particular, a novel layer-wise Transformer is designed to help fuse multimodal data by adding connection between each of the encoder and decoder layers. Numerical results show that the proposed models are superior to traditional communications in terms of the robustness to channels, computational complexity, transmission delay, and the task-execution performance at various task-specific metrics.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1179 · Cited by 25,545 · cited 1×
“, LSTM networks [36] and Gated Recurrent Units (GRU) networks [37].”
BackgroundBLEU: a method for automatic evaluation of machine translation
Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073135 · Cited by 21,344
Auto-Encoding Variational Bayes
Kingma, D. P., & Welling, M. (2013) · arXiv · doi:10.48550/arxiv.1312.6114 · Cited by 20,067
Goldsmith, A. (2005) · Cambridge University Press eBooks · doi:10.1017/cbo9780511841224 · Cited by 10,658
Dimensionality Reduction by Learning an Invariant Mapping
Hadsell, R., Chopra, S., & LeCun, Y. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2 (CVPR'06) · doi:10.1109/cvpr.2006.100 · Cited by 5,336
Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering
Anderson, P., He, X., Buehler, C., Teney, D., Johnson, M., Gould, S., & Zhang, L. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00636 · Cited by 4,948
Caltech-UCSD Birds-200-2011 Dataset
Wah, C. (2024) · CaltechAUTHORS (California Institute of Technology) · doi:10.57702/i9oeewf6 · Cited by 3,525
3D Object Representations for Fine-Grained Categorization
Krause, J., Stark, M., Deng, J., & Fei-Fei, L. (2013) · 2013 IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2013.77 · Cited by 3,496
An Overview of Massive MIMO: Benefits and Challenges
Lu, L., Li, G. Y., Swindlehurst, A. L., Ashikhmin, A., & Zhang, R. (2014) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2014.2317671 · Cited by 2,838
The Roadmap to 6G: AI Empowered Wireless Networks
Letaief, K. B., Chen, W., Shi, Y., Zhang, J., & Zhang, Y.-J. A. (2019) · IEEE Communications Magazine · doi:10.1109/mcom.2019.1900271 · Cited by 2,207
Self-citationStacked Attention Networks for Image Question Answering
Yang, Z., He, X., Gao, J., Deng, L., & Smola, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.10 · Cited by 1,998
DeepFashion: Powering Robust Clothes Recognition and Retrieval with Rich Annotations
Liu, Z., Luo, P., Qiu, S., Wang, X., & Tang, X. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.124 · Cited by 1,803
CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoning
Johnson, J., Hariharan, B., van der Maaten, L., Fei-Fei, L., Zitnick, C. L., & Girshick, R. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.215 · Cited by 1,781
Deep Metric Learning via Lifted Structured Feature Embedding
Song, H. O., Xiang, Y., Jegelka, S., & Savarese, S. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.434 · Cited by 1,630
Deep Learning Enabled Semantic Communication Systems
Xie, H., Qin, Z., Li, G. Y., & Juang, B.-H. (2021) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2021.3071210 · Cited by 1,602
Self-citationDeep Joint Source-Channel Coding for Wireless Image Transmission
Bourtsoulatze, E., Burth Kurka, D., & Gunduz, D. (2019) · IEEE Transactions on Cognitive Communications and Networking · doi:10.1109/tccn.2019.2919300 · Cited by 1,476
A review of internet of things for smart home: challenges and solutions
Risteska Stojkoska, B. L., & Trivodaliev, K. V. (2017) · Journal of Cleaner Production · doi:10.1016/j.jclepro.2016.10.006 · Cited by 1,397
Boehning, D., & Snyder, S. H. (2003) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.26.041002.131047 · Cited by 1,225
Deep multimodal learning: a survey on recent advances and trends
Ramachandram, D., & Taylor, G. W. (2017) · IEEE Signal Processing Magazine · doi:10.1109/msp.2017.2738401 · Cited by 1,139 · cited 1×
“…employ more than one user to serve one multimodal intelligent task, which is suitable for the emerging autonomous scenarios in daily life [19] and industry [20], i.e., autonomous checkout at retail stores [21], intelligent control at smart home [22], and human activity recognition in…”
BackgroundDeep Modular Co-Attention Networks for Visual Question Answering
Yu, Z., Yu, J., Cui, Y., Tao, D., & Tian, Q. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00644 · Cited by 964
Deep Image Retrieval: Learning Global Representations for Image Search
Gordo, A., Almazán, J., Revaud, J., & Larlus, D. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46466-4_15 · Cited by 787
Particular object retrieval with integral max-pooling of CNN activations
Tolias, G., Sicre, R., & Jégou, H. (2015) · arXiv · doi:10.48550/arxiv.1511.05879 · Cited by 744
Aggregating Local Deep Features for Image Retrieval
Yandex, A. B., & Lempitsky, V. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.150 · Cited by 674
Semantic Communication Systems for Speech Transmission
Weng, Z., & Qin, Z. (2021) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2021.3087240 · Cited by 663
Self-citationA Lite Distributed Semantic Communication System for Internet of Things
Xie, H., & Qin, Z. (2021) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2020.3036968 · Cited by 497
Self-citationDeepJSCC-f: Deep Joint Source-Channel Coding of Images With Feedback
Kurka, D. B., & Gunduz, D. (2020) · IEEE Journal on Selected Areas in Information Theory · doi:10.1109/jsait.2020.2987203 · Cited by 470
Deep Learning for Joint Source-Channel Coding of Text
Farsad, N., Rao, M., & Goldsmith, A. (2018) · 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2018.8461983 · Cited by 438
Semantics-Empowered Communication for Networked Intelligent Systems
Kountouris, M., & Pappas, N. (2021) · IEEE Communications Magazine · doi:10.1109/mcom.001.2000604 · Cited by 334
Multi-gigabit millimeter wave wireless communications for 5G: from fixed access to cellular networks
Wang, P., Li, Y., Song, L., & Vucetic, B. (2015) · IEEE Communications Magazine · doi:10.1109/mcom.2015.7010531 · Cited by 263
Propagation Modeling for Wireless Communications in the Terahertz Band
Han, C., & Chen, Y. (2018) · IEEE Communications Magazine · doi:10.1109/mcom.2018.1700898 · Cited by 255
Simple Baseline for Visual Question Answering
Zhou, B., Tian, Y., Sukhbaatar, S., Szlam, A. D., & Fergus, R. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1512.02167 · Cited by 254
Semantic Communications: Principles and Challenges
Qin, Z., Tao, X., Lü, J., Tong, W., & Li, G. Y. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2201.01389 · Cited by 254
Self-citationTask-Oriented Multi-User Semantic Communications for VQA
Xie, H., Qin, Z., & Li, G. Y. (2022) · IEEE Wireless Communications Letters · doi:10.1109/lwc.2021.3136045 · Cited by 243
Self-citationDeep Learning for Instance Retrieval: A Survey
Chen, W., Liu, Y., Wang, W., Bakker, E. M., Georgiou, T., Fieguth, P., Liu, L., & Lew, M. S. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3218591 · Cited by 201
(2020) · doi:10.1007/978-3-030-58586-0_27 · Cited by 156
Deep Learning-Constructed Joint Transmission-Recognition for Internet of Things
Lee, C., Lin, J.-W., Chen, P.-H., & Chang, Y.-C. (2019) · IEEE Access · doi:10.1109/access.2019.2920929 · Cited by 138
Training Vision Transformers for Image Retrieval
El-Nouby, A., Neverova, N., Laptev, I., & Jégou, H. (2021) · arXiv · doi:10.48550/arxiv.2102.05644 · Cited by 127
Encoding Source Language with Convolutional Neural Network for Machine Translation
Meng, F., Lu, Z., Wang, M., Li, H., Jiang, W., & Liu, Q. (2015) · Annual Meeting of the Association for Computational Linguistics · doi:10.3115/v1/p15-1003 · Cited by 120
WiFi and Vision Multimodal Learning for Accurate and Robust Device-Free Human Activity Recognition
Zou, H., Yang, J., Das, H., Liu, H., Zhou, Y., & Spanos, C. J. (2019) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2019.00056 · Cited by 85
A Survey of Multimodal Sensor Fusion for Passive RF and EO Information Integration
Vakil, A., Liu, J., Zulch, P. A., Blasch, E., Ewing, R. L., & Li, J. (2021) · IEEE Aerospace and Electronic Systems Magazine · doi:10.1109/maes.2020.3006410 · Cited by 69
Federated Learning for Audio Semantic Communication
Tong, H., Yang, Z., Wang, S., Hu, Y., Semiari, O., Saad, W., & Yin, C. (2021) · Frontiers in Communications and Networks · doi:10.3389/frcmn.2021.734402 · Cited by 55
Semantic Communications for Speech Recognition
Weng, Z., Qin, Z., & Li, G. Y. (2021) · Globecom · doi:10.1109/globecom46510.2021.9685250 · Cited by 55
Self-citationClustering-driven unsupervised deep hashing for image retrieval
Gu, Y., Wang, S., Zhang, H., Yao, Y., Yang, W., & Liu, L. (2019) · Neurocomputing · doi:10.1016/j.neucom.2019.08.050 · Cited by 45
Object-Location-Aware Hashing for Multi-Label Image Retrieval via Automatic Mask Learning
Huang, C., Yang, S., Pan, Y., & Lai, H. (2018) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2839522 · Cited by 41
Semantic-Aware Collaborative Deep Reinforcement Learning Over Wireless Cellular Networks
Lotfi, F., Semiari, O., & Saad, W. (2022) · International Conference on Communications · doi:10.1109/icc45855.2022.9839122 · Cited by 37
Ruiz, C., Falcão, J. D., & Zhang, P. (2019) · Adjunct ACM International Joint Conference on Pervasive and Ubiquitous Computing and ACM International Symposium on Wearable Computers (UbiComp/ISWC) · doi:10.1145/3341162.3345604 · Cited by 5
Wireless Image Retrieval at the Edge
Jankowski, M., Gunduz, D., & Mikolajczyk, K. (2021) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2020.3036955 · Cited by 3 · cited 1×
“[16] considered image based re-identification for person or cars as the communication task, in which two schemes (digital and analog) are proposed to improve the retrieval accuracy.”
Background
Cited by
Beyond Transmitting Bits: Context, Semantics, and Task-Oriented Communications
Gunduz, D., Qin, Z., Aguerri, I. E., Dhillon, H. S., Yang, Z., Yener, A., Wong, K. K., & Chae, C.-B. (2023) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2022.3223408 · Cited by 776
Self-citationSemantic Communications for Future Internet: Fundamentals, Applications, and Challenges
Yang, W., Du, H., Liew, Z. Q., Lim, W. Y. B., Xiong, Z., Niyato, D., Chi, X., Shen, X., & Miao, C. (2023) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2022.3223224 · Cited by 670
Xu, W., Yang, Z., Ng, D. W. K., Levorato, M., Eldar, Y. C., & Debbah, M. (2023) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2023.3239189 · Cited by 660
Wang, Y., Su, Z., Guo, S., Dai, M., Luan, T. H., & Liu, Y. (2023) · IEEE Internet of Things Journal · doi:10.1109/jiot.2023.3263909 · Cited by 357
Deep Learning Enabled Semantic Communications With Speech Recognition and Synthesis
Weng, Z., Qin, Z., Tao, X., Pan, C., Liu, G., & Li, G. Y. (2023) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2023.3240969 · Cited by 286
Self-citation
Metadata
- Authors
- Xie, H., Qin, Z., Tao, X., & Letaief, K. B.
- Venue
- IEEE Journal on Selected Areas in Communications
- Published
- 2022-09
- Volume
- 40
- Issue
- 9
- Pages
- 2584-2597
- ISSN
- 0733-8716 (ISSN-L)
- 1558-0008
- OpenAlex
- W4226115138
- Cited by
- 458
- References
- 48
- FWCI
- 24.09
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.