The DOI matches its Crossref record; 47 of the 47 references it cites were found in a registry or an index.
Learning Task-Oriented Communication for Edge Inference: An Information Bottleneck Approach
Open accessAbstract
This paper investigates task-oriented communication for edge inference, where a low-end edge device transmits the extracted feature vector of a local data sample to a powerful edge server for processing. It is critical to encode the data into aninformativeandcompactrepresentation for low-latency inference given the limited bandwidth. We propose a learning-based communication scheme that jointly optimizes feature extraction, source coding, and channel coding in a task-oriented manner, i.e., targeting the downstream inference task rather than data reconstruction. Specifically, we leverage an information bottleneck (IB) framework to formalize a rate-distortion tradeoff between the informativeness of the encoded feature and the inference performance. As the IB optimization is computationally prohibitive for the high-dimensional data, we adopt a variational approximation, namely the variational information bottleneck (VIB), to build a tractable upper bound. To reduce the communication overhead, we leverage a sparsity-inducing distribution as the variational prior for the VIB framework to sparsify the encoded feature vector. Furthermore, considering dynamic channel conditions in practical communication systems, we propose a variable-length feature encoding scheme based on dynamic neural networks to adaptively adjust the activated dimensions of the encoded feature to different channel conditions. Extensive experiments evidence that the proposed task-oriented communication system achieves a better rate-distortion tradeoff than baseline methods and significantly reduces the feature transmission latency in dynamic channel conditions.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
(2008) · Journal of Machine Learning Research · Cited by 33,051
Learning Multiple Layers of Features from Tiny Images
Krizhevsky, A. (2024) · doi:10.57702/zp44cu3g · Cited by 23,395
Auto-Encoding Variational Bayes
Kingma, D. P., & Welling, M. (2013) · arXiv · doi:10.48550/arxiv.1312.6114 · Cited by 20,067
Representation Learning: A Review and New Perspectives
Bengio, Y., Courville, A., & Vincent, P. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.50 · Cited by 13,001
Speech recognition with deep recurrent neural networks
Graves, A., Mohamed, A.-R., & Hinton, G. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6638947 · Cited by 8,893
A unified architecture for natural language processing: deep neural networks with multitask learning
Collobert, R., & Weston, J. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390177 · Cited by 5,058
Channel Coding Rate in the Finite Blocklength Regime
Polyanskiy, Y., Poor, H. V., & Verdu, S. (2010) · IEEE Transactions on Information Theory · doi:10.1109/tit.2010.2043769 · Cited by 3,885
Deep learning for computer vision: A brief review
Voulodimos, A., Doulamis, N., Doulamis, A., & Protopapadakis, E. (2018) · Computational Intelligence and Neuroscience · doi:10.1155/2018/7068349 · Cited by 3,403
An Introduction to Deep Learning for the Physical Layer
O'Shea, T., & Hoydis, J. (2017) · IEEE Transactions on Cognitive Communications and Networking · doi:10.1109/tccn.2017.2758370 · Cited by 2,980
The information bottleneck method
Tishby, N., Pereira, F. C., & Bialek, W. (2000) · arXiv · doi:10.48550/arxiv.physics/0004057 · Cited by 2,626 · cited 4×
“To address this limitation and improve the inference performance, in this paper, we propose a task-oriented communication principle for edge inference and develop an innovative learning-driven approach under the framework of information bottleneck (IB) [20].”
InfluentialThe Roadmap to 6G: AI Empowered Wireless Networks
Letaief, K. B., Chen, W., Shi, Y., Zhang, J., & Zhang, Y.-J. A. (2019) · IEEE Communications Magazine · doi:10.1109/mcom.2019.1900271 · Cited by 2,207
Self-citationDeep Joint Source-Channel Coding for Wireless Image Transmission
Bourtsoulatze, E., Burth Kurka, D., & Gunduz, D. (2019) · IEEE Transactions on Cognitive Communications and Networking · doi:10.1109/tccn.2019.2919300 · Cited by 1,476
Artificial Neural Networks-Based Machine Learning for Wireless Networks: A Tutorial
Chen, M., Challita, U., Saad, W., Yin, C., & Debbah, M. (2019) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2019.2926625 · Cited by 1,117
Elements of Information Theory
(2013) · Wireless Communications · doi:10.1002/9781118576618.ch7 · Cited by 999
Deep Learning Based Communication Over the Air
Dorner, S., Cammerer, S., Hoydis, J., & Brink, S. T. (2018) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2017.2784180 · Cited by 859
Once-for-All: Train One Network and Specialize it for Efficient Deployment
Cai, H., Gan, C., Wang, T., Zhang, Z., & Han, S. (2019) · arXiv · doi:10.48550/arxiv.1908.09791 · Cited by 845
Neurosurgeon: Collaborative Intelligence Between the Cloud and Mobile Edge
Kang, Y., Hauswald, J., Gao, C., Rovinski, A., Mudge, T., Mars, J., & Tang, L. (2017) · ACM SIGARCH Computer Architecture News · doi:10.1145/3093337.3037698 · Cited by 806
Neurosurgeon: Collaborative Intelligence Between the Cloud and Mobile Edge
Kang, Y., Hauswald, J., Gao, C., Rovinski, A., Mudge, T., Mars, J., & Tang, L. (2017) · Proceedings of the Twenty-Second International Conference on Architectural Support for Programming Languages and Operating Systems · doi:10.1145/3037697.3037698 · Cited by 673
SkipNet: Learning Dynamic Routing in Convolutional Networks
Wang, X., Yu, F., Dou, Z.-Y., Darrell, T., & Gonzalez, J. E. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01261-8_25 · Cited by 625
Toward an Intelligent Edge: Wireless Communication Meets Machine Learning
Zhu, G., Liu, D., Du, Y., You, C., Zhang, J., & Huang, K. (2020) · IEEE Communications Magazine · doi:10.1109/mcom.001.1900103 · Cited by 606
Self-citationEdge Assisted Real-time Object Detection for Mobile Augmented Reality
Liu, L., Li, H., & Gruteser, M. (2019) · The 25th Annual International Conference on Mobile Computing and Networking · doi:10.1145/3300061.3300116 · Cited by 552
Samuel, N., Diskin, T., & Wiesel, A. (2019) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2019.2899805 · Cited by 515
Event-Based Vision Meets Deep Learning on Steering Prediction for Self-Driving Cars
Maqueda, A. I., Loquercio, A., Gallego, G., Garcia, N., & Scaramuzza, D. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00568 · Cited by 514
Event-Based Vision Meets Deep Learning on Steering Prediction for Self-Driving Cars
Maqueda, A. I., Loquercio, A., Gallego, G., Garcia, N. N., & Scaramuzza, D. (2018) · Zurich Open Repository and Archive (University of Zurich) · doi:10.5167/uzh-175994 · Cited by 493
Communication-efficient edge ai: algorithms and systems
Shi, Y., Yang, K., Jiang, T., Zhang, J., & Letaief, K. B. (2020) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2020.3007787 · Cited by 488
Self-citationBlockDrop: Dynamic Inference Paths in Residual Networks
Wu, Z., Nagarajan, T., Kumar, A., Rennie, S., Davis, L. S., Grauman, K., & Feris, R. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00919 · Cited by 465
Deep Learning for Joint Source-Channel Coding of Text
Farsad, N., Rao, M., & Goldsmith, A. (2018) · 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2018.8461983 · Cited by 438
Edge Intelligence: On-Demand Deep Learning Model Co-Inference with Device-Edge Synergy
Li, E., Zhou, Z., & Chen, X. (2018) · Proceedings of the 2018 Workshop on Mobile Edge Communications · doi:10.1145/3229556.3229562 · Cited by 385
Shen, Y., Shi, Y., Zhang, J., & Letaief, K. B. (2021) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2020.3036965 · Cited by 367
Self-citationInformation dropout: learning optimal representations through noisy computation
Achille, A., & Soatto, S. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2784440 · Cited by 356
Information transmission with additional noise
Dobrushin, R., & Tsybakov, B. (1962) · IEEE Transactions on Information Theory · doi:10.1109/tit.1962.1057738 · Cited by 270
Yu, J., Yang, L., Xu, N., Yang, J., & Huang, T. S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.08928 · Cited by 216
BottleNet++: An End-to-End Approach for Feature Compression in Device-Edge Co-Inference Systems
Shao, J., & Zhang, J. (2020) · 2020 IEEE International Conference on Communications Workshops (ICC Workshops) · doi:10.1109/iccworkshops49005.2020.9145068 · Cited by 200
Self-citationThe Information Bottleneck Problem and its Applications in Machine Learning
Goldfeld, Z., & Polyanskiy, Y. (2020) · IEEE Journal on Selected Areas in Information Theory · doi:10.1109/jsait.2020.2991561 · Cited by 165
Communication-Computation Trade-off in Resource-Constrained Edge Inference
Shao, J., & Zhang, J. (2020) · IEEE Communications Magazine · doi:10.1109/mcom.001.2000373 · Cited by 162
Self-citationJALAD: Joint Accuracy-And Latency-Aware Deep Structure Decoupling for Edge-Cloud Execution
Li, H., Hu, C., Jiang, J., Wang, Z., Wen, Y., & Zhu, W. (2018) · 2018 IEEE 24th International Conference on Parallel and Distributed Systems (ICPADS) · doi:10.1109/padsw.2018.8645013 · Cited by 160
Zaidi, A., Aguerri, I. E., & Shamai, S. S. (2020) · Entropy · doi:10.3390/e22020151 · Cited by 123
You Look Twice: GaterNet for Dynamic Filter Selection in CNNs
Chen, Z., Li, Y., Samy Bengio, A., & Si, S. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00939 · Cited by 109
Improving Device-Edge Cooperative Inference of Deep Learning via 2-Step Pruning
Shi, W., Hou, Y., Zhou, S., Niu, Z., Zhang, Y., & Geng, L. (2019) · IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS) · doi:10.1109/infocomwkshps47286.2019.9093772 · Cited by 104
Nonparametric density estimation for high‐dimensional data—Algorithms and applications
Wang, Z., & Scott, D. W. (2019) · Wiley Interdisciplinary Reviews Computational Statistics · doi:10.1002/wics.1461 · Cited by 96
Deep Joint Source-Channel Coding for Wireless Image Retrieval
Jankowski, M., Gündüz, D., & Mikolajczyk, K. (2020) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp40776.2020.9054078 · Cited by 70
Branchy-GNN: A Device-Edge Co-Inference Framework for Efficient Point Cloud Processing
Shao, J., Zhang, H., Mao, Y., & Zhang, J. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9414831 · Cited by 63
Self-citationPredictive View Generation to Enable Mobile 360-degree and VR Experiences
Hou, X., Dey, S., Zhang, J., & Budagavi, M. (2018) · doi:10.1145/3229625.3229629 · Cited by 49
Self-citationWireless Image Retrieval at the Edge
Jankowski, M., Gunduz, D., & Mikolajczyk, K. (2021) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2020.3036955 · Cited by 3
Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations
Hubara, I., Courbariaux, M., Soudry, D., El-Yaniv, R., & Bengio, Y. (2016) · arXiv · doi:10.48550/arxiv.1609.07061
Research works printed but not linked: 8
- choi, 2019, Neural joint source-channel coding, Proc Int Conf Mach Learn, 1182
- alemi, 2017, Deep variational information bottleneck, Proc Int Conf Learn Represent, 1
- wen, 2016, Learning structured sparsity in deep neural networks, Proc Int Conf Neural Inf Process, 2082
- 10.1109/ISIT.2004.1365385 · doi:10.1109/isit.2004.1365385
- kingma, 2015, Variational dropout and the local reparameterization trick, Proc Adv Neural Inf Process Syst, 2575
- 10.1109/MSPEC.2020.9078454 · doi:10.1109/mspec.2020.9078454
- le, 2015, Tiny imagenet visual recognition challenge
- molchanov, 2017, Variational dropout sparsifies deep neural networks, Proc Int Conf Mach Learn, 2498
Cited by
Edge artificial intelligence for 6g: vision, enabling technologies, and applications
Letaief, K. B., Shi, Y., Lu, J., & Lu, J. (2022) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2021.3126076 · Cited by 829
Beyond Transmitting Bits: Context, Semantics, and Task-Oriented Communications
Gunduz, D., Qin, Z., Aguerri, I. E., Dhillon, H. S., Yang, Z., Yener, A., Wong, K. K., & Chae, C.-B. (2023) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2022.3223408 · Cited by 776
Xu, W., Yang, Z., Ng, D. W. K., Levorato, M., Eldar, Y. C., & Debbah, M. (2023) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2023.3239189 · Cited by 660
Deep Learning Enabled Semantic Communications With Speech Recognition and Synthesis
Weng, Z., Qin, Z., Tao, X., Pan, C., Liu, G., & Li, G. Y. (2023) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2023.3240969 · Cited by 286
Robust Semantic Communications With Masked VQ-VAE Enabled Codebook
Hu, Q., Zhang, G., Qin, Z., Cai, Y., Yu, G., & Li, G. Y. (2023) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2023.3265201 · Cited by 282
Metadata
- Authors
- Shao, J., Mao, Y., & Zhang, J.
- Venue
- IEEE Journal on Selected Areas in Communications
- Published
- 2022-01
- Volume
- 40
- Issue
- 1
- Pages
- 197-211
- ISSN
- 0733-8716 (ISSN-L)
- 1558-0008
- OpenAlex
- W3214462705
- Cited by
- 338
- References
- 47
- FWCI
- 19.33
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.