Mobile Edge Intelligence for Large Language Models: A Contemporary Survey
Open accessThe DOI matches its Crossref record; 39 of the 39 references it cites were found in a registry or an index.
Abstract
On-device large language models (LLMs), referring to running LLMs on edge devices, have raised considerable interest owing to their superior privacy, reduced latency, and bandwidth saving. Nonetheless, the capabilities of on-device LLMs are intrinsically constrained by the limited capacity of edge devices compared to the much more powerful cloud centers. To bridge the gap between cloud-based and on-device AI, mobile edge intelligence (MEI) presents a viable solution to this problem by provisioning AI capabilities within the edge of mobile networks with improved privacy and latency relative to cloud computing. MEI sits between on-device AI and cloud-based AI, featuring wireless communications and more powerful computing resources than end devices. This article provides a contemporary survey on harnessing MEI for LLMs. We first cover the preliminaries of LLMs, starting with LLMs and MEI, followed by resource-efficient LLM techniques. We then illustrate several killer applications to demonstrate the need for deploying LLMs at the network edge and present an architectural overview of MEI for LLMs (MEI4LLM). Subsequently, we delve into various aspects of MEI4LLM, extensively covering edge LLM caching and delivery, edge LLM training, and edge LLM inference. Finally, we identify future research opportunities. We aim to inspire researchers in the field to leverage mobile edge computing to facilitate LLM deployment in close proximity to users, thereby unleashing the potential of LLMs across various privacy-and delay-sensitive applications.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
End-to-End Object Detection with Transformers
Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., & Zagoruyko, S. (2020) · arXiv · doi:10.48550/arxiv.2005.12872 · Cited by 18,253
Distilling the Knowledge in a Neural Network
Hinton, G., Vinyals, O., & Dean, J. (2015) · arXiv · doi:10.48550/arxiv.1503.02531 · Cited by 14,204
HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units
Hsu, W.-N., Bolte, B., Tsai, Y.-H. H., Lakhotia, K., Salakhutdinov, R., & Mohamed, A. (2021) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2021.3122291 · Cited by 2,983
Graves, A. (2012) · Studies in Computational Intelligence · doi:10.1007/978-3-642-24797-2_4 · Cited by 2,727
Llama 2: Open Foundation and Fine-Tuned Chat Models
Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al. (2023) · arXiv · doi:10.48550/arxiv.2307.09288 · Cited by 2,604
Parameter-efficient transfer learning for NLP
Houlsby, N., Giurgiu, A., Jastrzebski, S., Morrone, B., de Laroussilhe, Q., Gesmundo, A., Attariyan, M., & Gelly, S. (2019) · arXiv · doi:10.48550/arxiv.1902.00751 · Cited by 2,270
A Survey of Large Language Models
Zhao, W. X., Zhou, K., Li, J., Tang, T., Dong, Z., Hou, Y., Zhang, B., Min, Y., Zhang, J., Liu, P., et al. (2026) · Frontiers of Computer Science · doi:10.1007/s11704-026-60308-3 · Cited by 2,093
mT5: A Massively Multilingual Pre-trained Text-to-Text Transformer
Xue, L., Constant, N., Roberts, A., Kale, M., Al-Rfou, R., Siddhant, A., Barua, A., & Raffel, C. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.41 · Cited by 1,629
Neural Machine Translation of Rare Words with Subword Units
Sennrich, R., Haddow, B., & Birch, A. (2016) · Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p16-1162 · Cited by 974 · cited 1×
“First, input data, such as words or sentences, are segmented into token sequences through a process called tokenization, using tokenizers like WordPiece [56], [57] or Byte-Pair Encoding [51], [58].”
Neural Machine Translation of Rare Words with Subword Units
Sennrich, R., Haddow, B., & Birch, A. (2015) · arXiv · doi:10.48550/arxiv.1508.07909 · Cited by 950
(2020) · doi:10.1109/sc41405.2020.00024 · Cited by 928
Japanese and Korean voice search
Schuster, M., & Nakajima, K. (2012) · 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2012.6289079 · Cited by 862
A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Models
Fan, W., Ding, Y., Ning, L., Wang, S., Li, H., Yin, D., Chua, T.-S., & Li, Q. (2024) · Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining · doi:10.1145/3637528.3671470 · Cited by 827 · cited 1×
“In addition, considering RAG [195], [196], external knowledge sources should also be cached at the network edge, ensuring timely acquisition of the data/knowledge for an LLM.”
Split learning for health: Distributed deep learning without sharing raw patient data
Vepakomma, P., Gupta, O., Swedish, T., & Raskar, R. (2018) · arXiv · doi:10.48550/arxiv.1812.00564 · Cited by 513 · cited 1×
“Since its invention, SL has been applied to various scenarios, such as healthcare [251], [252].”
Carbon Emissions and Large Neural Network Training
Patterson, D., Gonzalez, J., Le, Q., Liang, C., Munguia, L.-M., Rothchild, D., So, D., Texier, M., & Dean, J. (2021) · arXiv · doi:10.48550/arxiv.2104.10350 · Cited by 486
Deep Learning for Smart Industry: Efficient Manufacture Inspection System With Fog Computing
Li, L., Ota, K., & Dong, M. (2018) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2018.2842821 · Cited by 466 · cited 1×
“…Things (IoT) applications, Microsoft “Azure IoT Edge”, Google “Cloud IoT”, Amazon “Web Services IoT”, and NVIDIA “EGX” provide edge AI platforms to bring real-time AI services across a wide range of applications, spanning from live video analytics [124], smart home [125], and industrial IoT [126].”
Benchmarking Large Language Models in Retrieval-Augmented Generation
Chen, J., Lin, H., Han, X., & Sun, L. (2023) · arXiv · doi:10.48550/arxiv.2309.01431 · Cited by 455 · cited 1×
“In addition, considering RAG [195], [196], external knowledge sources should also be cached at the network edge, ensuring timely acquisition of the data/knowledge for an LLM.”
Dynamic Adaptive DNN Surgery for Inference Acceleration on the Edge
Hu, C., Bao, W., Wang, D., & Liu, F. (2019) · IEEE INFOCOM 2019 - IEEE Conference on Computer Communications · doi:10.1109/infocom.2019.8737614 · Cited by 416 · cited 1×
“Similar problems have been considered for multi-hop split inference [235], [236], where the model splitting and/or placement is often mapped to a graph, which can be further mapped to shortest path problem [236] or min-cut problem [235].”
Towards Expert-Level Medical Question Answering with Large Language Models
Singhal, K., Tu, T., Gottweis, J., Sayres, R., Wulczyn, E., Hou, L., Clark, K., Pfohl, S., Cole-Lewis, H., Neal, D., et al. (2023) · arXiv · doi:10.48550/arxiv.2305.09617 · Cited by 338
Large Language Models: A Survey
Minaee, S., Mikolov, T., Nikzad, N., Chenaghlu, M., Socher, R., Amatriain, X., & Gao, J. (2024) · arXiv · doi:10.48550/arxiv.2402.06196 · Cited by 325 · cited 2×
“These models are extensively pre-trained on massive datasets using deep learning techniques [10], [65].”
(2021) · doi:10.1145/3458817.3476205 · Cited by 257
GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers
Frantar, E., Ashkboos, S., Hoefler, T., & Alistarh, D. (2022) · arXiv · doi:10.48550/arxiv.2210.17323 · Cited by 197
A Review of Sparse Expert Models in Deep Learning
Fedus, W., Dean, J., & Zoph, B. (2022) · arXiv · doi:10.48550/arxiv.2209.01667 · Cited by 196
LLMLingua: Compressing Prompts for Accelerated Inference of Large Language Models
Jiang, H., Wu, Q., Lin, C.-Y., Yang, Y., & Qiu, L. (2023) · arXiv · Cited by 168 · cited 1×
“It has been shown that input token pruning does not introduce significant inference accuracy loss for LLMs [266].”
OpenEI: An Open Framework for Edge Intelligence
Zhang, X., Wang, Y., Lu, S., Liu, L., xu, L., & Shi, W. (2019) · 2019 IEEE 39th International Conference on Distributed Computing Systems (ICDCS) · doi:10.1109/icdcs.2019.00182 · Cited by 139 · cited 1×
“This greatly enhances both the efficiency and speed of data processing while enhancing data privacy [117], [118], [119].”
Not All Patches are What You Need: Expediting Vision Transformers via Token Reorganizations
Liang, Y., Ge, C., Tong, Z., Song, Y., Wang, J., & Xie, P. (2022) · arXiv · doi:10.48550/arxiv.2202.07800 · Cited by 131
Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Survey
Han, Z., Gao, C., Liu, J., Zhang, J., & Zhang, S. Q. (2024) · arXiv · doi:10.48550/arxiv.2403.14608 · Cited by 129
Reducing the Carbon Impact of Generative AI Inference (today and in 2035)
Chien, A. A., Lin, L., Nguyen, H., Rao, V., Sharma, T., & Wijayawardana, R. (2023) · Proceedings of the 2nd Workshop on Sustainable Computer Systems · doi:10.1145/3604930.3605705 · Cited by 128
(2023) · doi:10.52202/075280-1506 · Cited by 113
AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration
Lin, J., Tang, J., Tang, H., Yang, S., Chen, W.-M., Wang, W.-C., Xiao, G., Dang, X., Gan, C., & Han, S. (2023) · arXiv · doi:10.48550/arxiv.2306.00978 · Cited by 103 · cited 5×
“For example, weights corresponding to larger activations or higher quantization errors can be quantized in high bitwidth before downloading, which can preserve the performance of the quantized LLMs [131], [207], [208].”
InfluentialCompressing Context to Enhance Inference Efficiency of Large Language Models
Li, Y., Dong, B., Lin, C., & Guerin, F. (2023) · arXiv · Cited by 88 · cited 1×
“By removing some unimportant tokens from input text sequences and images before inference, the sizes of data that need to be offloaded to edge servers are reduced [265], [279], [280].”
A Survey of Resource-efficient LLM and Multimodal Foundation Models
Xu, M., Yin, W., Cai, D., Yi, R., Xu, D., Wang, Q., Wu, B., Zhao, Y., Yang, C., Wang, S., et al. (2024) · arXiv · doi:10.48550/arxiv.2401.08092 · Cited by 44
Poisoning Language Models During Instruction Tuning
Wan, A., Wallace, E., Shen, S., & Klein, D. (2023) · arXiv · doi:10.48550/arxiv.2305.00944 · Cited by 42
He, K., Mao, R., Lin, Q., Ruan, Y., Lan, X., Feng, M., & Cambria, E. (2023) · arXiv · doi:10.48550/arxiv.2310.05694 · Cited by 27 · cited 1×
“Apart from this, healthcare LLMs can also assist in medical question answering, diagnosis, treatment, and medical report generation [36].”
Parameter-Efficient Transfer Learning with Diff Pruning
Guo, D., Rush, A. M., & Kim, Y. (2020) · arXiv · doi:10.48550/arxiv.2012.07463 · Cited by 20
Hybrid LLM: Cost-Efficient and Quality-Aware Query Routing
Ding, D., Mallick, A., Wang, C., Sim, R., Mukherjee, S., Ruhle, V., Lakshmanan, L. V. S., & Awadallah, A. H. (2024) · arXiv · doi:10.48550/arxiv.2404.14618 · Cited by 12
Teach LLMs to Phish: Stealing Private Information from Language Models
Panda, A., Choquette-Choo, C. A., Zhang, Z., Yang, Y., & Mittal, P. (2024) · arXiv · doi:10.48550/arxiv.2403.00871 · Cited by 5
Cited by
From Edge Transformer to IoT Decisions: Offloaded Embeddings for Lightweight Intrusion Detection
Adjewa, F., Esseghir, M., & Merghem-Boulahia, L. (2026) · Sensors · doi:10.3390/s26020356 · Cited by 23
“In [32], authors covered strategies such as edge LLM caching, gradient checkpointing, and attention optimization, which are essential for deploying LLMs in low-latency, privacy-sensitive applications.”
Metadata
- Authors
- Qu, G., Chen, Q., Wei, W., Lin, Z., Chen, X., & Huang, K.
- Venue
- IEEE Communications Surveys & Tutorials
- Published
- 2025-12
- Volume
- 27
- Issue
- 6
- Pages
- 3820-3860
- ISSN
- 1553-877X (ISSN-L)
- 2373-745X
- OpenAlex
- W4406207083
- Cited by
- 235
- References
- 39
- FWCI
- 102.27
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.