The DOI matches its Crossref record; 41 of the 41 references it cites were found in a registry or an index.
Optimal and Scalable Caching for 5G Using Reinforcement Learning of Space-Time Popularities
Open accessAbstract
Small basestations (SBs) equipped with caching units have potential to handle the unprecedented demand growth in heterogeneous networks. Through low-rate, backhaul connections with the backbone, SBs can prefetch popular files during off-peak traffic hours, and service them to the edge at peak periods. To intelligently prefetch, each SB must learn what and when to cache, while taking into account SB memory limitations, the massive number of available contents, the unknown popularity profiles, as well as the space-time popularity dynamics of user file requests. In this paper, local and global Markov processes model user requests, and a reinforcement learning (RL) framework is put forth for finding the optimal caching policy when the transition probabilities involved are unknown. Joint consideration of global and local popularity demands along with cache-refreshing costs allow for a simple, yet practical asynchronous caching approach. The novel RL-based caching relies on a Q-learning algorithm to implement the optimal policy in an online fashion, thus, enabling the cache control unit at the SB to learn, track, and possibly adapt to the underlying dynamics. To endow the algorithm with scalability, a linear function approximation of the proposed Q-learning scheme is introduced, offering faster convergence as well as reduced complexity and memory requirements. Numerical tests corroborate the merits of the proposed approach in various realistic settings.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“But [7] is based on Q-Learning, which is not feasible for the practical MEC system where the stateaction space is tremendous”
From In-Edge AI: Intelligentizing Mobile Edge Computing, Caching and Communication by Federated Learning (2019) · cites this paper as [7]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. (2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 25,268
Artificial intelligence: a modern approach
Kumar, D. A., Balasubramanian, S., Byeon, D. H., & Manerkar, P. G. V. (1995) · Choice Reviews Online · doi:10.5860/choice.33-1577 · Cited by 22,191
Watkins, C., & Dayan, P. (1992) · Machine-mediated learning · doi:10.1023/a:1022676722315 · Cited by 14,277
Continuous control with deep reinforcement learning
Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., & Wierstra, D. (2015) · arXiv · doi:10.48550/arxiv.1509.02971 · Cited by 10,782
Artificial intelligence: A modern approach
Nilsson, N. J. (1996) · Artificial Intelligence · doi:10.1016/0004-3702(96)00007-0 · Cited by 10,730
Watkins, C. J. C. H., & Dayan, P. (1992) · Machine Learning · doi:10.1007/bf00992698 · Cited by 9,083
Reinforcement Learning: An Introduction
Johnson, J. D., Li, J., & Chen, Z. (2000) · Neurocomputing · doi:10.1016/s0925-2312(00)00324-6 · Cited by 8,825
Andrews, J. G., Buzzi, S., Choi, W., Hanly, S. V., Lozano, A., Soong, A. C. K., & Zhang, J. C. (2014) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2014.2328098 · Cited by 8,106
Web caching and Zipf-like distributions: evidence and implications
Breslau, L., Cao, P., Fan, L., Phillips, G., & Shenker, S. J. (1999) · IEEE Conference on Computer Communications (INFOCOM) · doi:10.1109/infcom.1999.749260 · Cited by 3,485
Device-to-device communication as an underlay to LTE-advanced networks
Doppler, K., Rinne, M., Wijting, C., Ribeiro, C. B., & Hugl, K. (2009) · IEEE Communications Magazine · doi:10.1109/mcom.2009.5350367 · Cited by 1,904
Flagged first, then by citations
Research works printed but not linked: 1
- lillicrap, 2016, Proc Int Conf Learn Representations
Cited by
Sorted by citations
Items 201–255 of 255
User Preference and Activity Aware Content Sharing in Wireless D2D Caching Networks
Qi, Y., Luo, J., Gao, L. X., Zheng, F., & Yu, L. C. (2020) · IEEE/CIC International Conference on Communications in China (ICCC) · doi:10.1109/iccc49849.2020.9238810 · Cited by 2
Yu, H., Qian, X., Qin, G., Ren, J., & Wang, X. (2023) · Communications in computer and information science · doi:10.1007/978-981-19-9697-9_48 · Cited by 2
Garg, N., Liu, Z., & Ratnarajah, T. (2024) · IEEE Access · doi:10.1109/access.2024.3359036 · Cited by 2
A Fault-Tolerant Mobility-Aware Caching Method in Edge Computing
Ma, Y., Zhao, H., Guo, K., Xia, Y., Wang, X., Niu, X., Zhu, D., & Dong, Y. (2024) · Computer Modeling in Engineering & Sciences · doi:10.32604/cmes.2024.048759 · Cited by 2
Proactive Edge Caching with LSTM-based Popularity Prediction
Demirci, İ., & Korçak, Ö. (2024) · International Balkan Conference on Communications and Networking (BalkanCom) · doi:10.1109/balkancom61808.2024.10557165 · Cited by 2
Singh, A., Kumar, A., & Chauhan, B. K. (2022) · Lecture notes in networks and systems · doi:10.1007/978-981-16-8763-1_19 · Cited by 2
Group Mobility-Based Optimization of Cache Content in Wireless Device-to-Device Networks
Naguib, K. M., Ali, A. S., & Mahmoud, K. R. (2018) · International Computer Engineering Conference · doi:10.1109/icenco.2018.8636120 · Cited by 2
Active Popularity Learning with Cache Hit Ratio Guarantees using a Matrix Completion Committee
Bommaraveni, S., Vu, T. X., Chatzinotas, S., & Ottersten, B. (2020) · IEEE International Symposium on Personal, Indoor and Mobile Radio Communications (PIMRC) · doi:10.1109/pimrc48278.2020.9217322 · Cited by 2
Modified reinforcement learning based- caching system for mobile edge computing
Mehamel, S., Bouzefrane, S., Banerjee, S., Daoui, M., & Bălaş, V. E. (2020) · Intelligent Decision Technologies · doi:10.3233/idt-190152 · Cited by 2
Caching deployment based on energy efficiency in device-to-device cooperative networks
Wang, W., Li, H., Liu, Y., Cheng, W., & Qin, H. (2020) · International Journal of Distributed Sensor Networks · doi:10.1177/1550147720984659 · Cited by 2
On the Exploration and Exploitation Trade-off in Cooperative Caching-enabled Networks
Bommaraveni, S., Gautam, S., Vu, T. X., & Chatzinotas, S. (2021) · IEEE Wireless Communications and Networking Conference Workshops (WCNCW) · doi:10.1109/wcncw49093.2021.9419984 · Cited by 2
Cache content placement in the presence of fictitious requests in mmWave 5G IAB networks
Nazarifard, F. S. H., Rashidi, Z., & Hakami, V. (2024) · Ad Hoc Networks · doi:10.1016/j.adhoc.2024.103563 · Cited by 2
A rolling-horizon dynamic programming approach for collaborative caching
Yang, X., & Thomos, N. (2019) · Open Access at Essex (University of Essex) · doi:10.48550/arxiv.1907.13516 · Cited by 2
Learning to Cache With No RegretsPreprint
Paschos, G. S., Destounis, A., Vigneri, L., & Iosifidis, G. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.09849 · Cited by 2
Deep Learning based Channel Estimation Utilizing Various 5G Channel Models
Thakur, R., & Nigam, V. S. (2025) · International Conference on Engineering Innovations and Technologies (ICoEIT) · doi:10.1109/icoeit63558.2025.11210876 · Cited by 1
Caching Contents with Varying Popularity Using Restless Bandits
Pavamana, K. J., & Singh, C. (2024) · Lecture notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering · doi:10.1007/978-3-031-48885-6_9 · Cited by 1
A Feature-Based Bayesian Method for Content Popularity Prediction in Edge-Caching Networks
Mehrizi, S., Tsakmalis, A., Chatzinotas, S., & Ottersten, B. (2019) · IEEE Wireless Communications and Networking Conference · doi:10.1109/wcnc.2019.8885590 · Cited by 1
Zhang, Y., & Xu, K. (2020) · Springer eBooks · doi:10.1007/978-981-15-0138-8_7 · Cited by 1
Maximizing Hit Ratio in Finite-Buffer Caching with Request Delay Information: An MDP Approach
Hui, H., & Chen, W. (2019) · Globecom · doi:10.1109/globecom38437.2019.9013571 · Cited by 1
Linear Approximation based Q-Learning for Edge Caching in Massive MIMO Networks
Garg, N., Sellathurai, M., & Ratnarajah, T. (2019) · Asilomar Conference on Signals, Systems and Computers (ACSSC) · doi:10.1109/ieeeconf44664.2019.9048737 · Cited by 1
Learning-Based Content Caching and User Clustering: A Deep Deterministic Policy Gradient Approach
Chan, K.-L., & Chien, F. (2020) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp40776.2020.9053382 · Cited by 1
Research on Discrete Artificial Bee Colony Cache Strategy of UAV Edge Network
Hong, Y., Zhang, Y., & Fan, S. (2022) · Processes · doi:10.3390/pr10091838 · Cited by 1
Overview of Edge Intelligence and Blockchain
Wang, X., Qiu, C., Ren, X., Xiong, Z., Leung, V. C. M., & Niyato, D. T. (2022) · Wireless networks · doi:10.1007/978-3-031-10186-1_2 · Cited by 1
2018 Index IEEE Journal of Selected Topics in Signal Processing Vol. 12
(2018) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2018.2886649 · Cited by 1
A Bayesian Poisson–Gaussian Process Model for Popularity Learning in Edge-Caching Networks
Mehrizi, S., Tsakmalis, A., Chatzinotas, S., & Ottersten, B. (2019) · IEEE Access · doi:10.1109/access.2019.2927494
Dynamic Content Updates in Heterogeneous Wireless NetworksPreprint
Abad, M. S. H., Özfatura, E., Erçetin, Ö., & Gündüz, D. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.09445
Distributed Network Caching via Dynamic ProgrammingPreprint
Sadeghi, A., Marques, A. G., & Giannakis, G. B. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.07121
Self-citationCooperation and Resource Allocation in Wireless Networking towards the IoT
Avgouleas, I. (2019) · Linköping studies in science and technology. Dissertations · doi:10.3384/diss.diva-161732
Zhang, Y., & Xu, K. (2020) · Springer eBooks · doi:10.1007/978-981-15-0138-8_8
Joint User Clustering and Content Caching with Heterogeneous User Content Preferences
Chiu, F., Kuo, T., Chien, F., Huang, W.-J., & Chang, M.-K. C. (2019) · Asilomar Conference on Signals, Systems and Computers (ACSSC) · doi:10.1109/ieeeconf44664.2019.9048847
A Transferable Edge Caching Method Based On Reinforcement Learning for Dense Small Cell Network
Hu, L., Fan, S., & Tian, H. (2020) · IEEE International Symposium on Personal, Indoor and Mobile Radio Communications (PIMRC) · doi:10.1109/pimrc48278.2020.9217130
Random-Mode Frank-Wolfe Algorithm for Tensor Completion in Wireless Edge Caching.Preprint
Garg, N., & Ratnarajah, T. (2021) · arXiv (Cornell University)
Obstacles in grid rolling out from 4G to 5G in Iraq
Al-Musawi, M. T. H., & Wahab, A. A. A. (2021) · Materials Today Proceedings · doi:10.1016/j.matpr.2021.08.313
Robust, Deep, and Reinforcement Learning for Management of Communication and Power NetworksPreprint
Sadeghi, A. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2202.05395
Self-citationOnline Tensor Completion Based Prediction For Wireless Edge Caching.Preprint
Garg, N., & Ratnarajah, T. (2021) · arXiv (Cornell University)
Glisic, S., & Lorenzo, B. (2022) · Wiley eBooks · doi:10.1002/9781119790327.ch7
Towards Optimising Fog Caching Using Deep Reinforcement Learning
Tajioue, M. A., Moutai, F. Z., Azzouzi, S., & Charaf, M. E. H. (2022) · IEEE International Conference on Electronics, Control, Optimization and Computer Science (ICECOCS) · doi:10.1109/icecocs55148.2022.9983128
RECC: A Relationship-Enhanced Content Caching Algorithm Using Deep Reinforcement Learning
Ren, J., Zhang, H., Zhou, X., & Zhu, M. (2022) · Asian Conference on Artificial Intelligence Technology (ACAIT) · doi:10.1109/acait56212.2022.10137967
Factorization Q‐Learning Initialization for Parameter Optimization in Cellular Networks
Zeng, B., Zhong, Y., Niu, X., & Tang, J. (2022) · Wireless Communications and Mobile Computing · doi:10.1155/2022/6538397
A Multi-Agent Deep Reinforcement Learning-Based Approach to Mobility-Aware Caching
Zhao, H., Shao, S., Ma, Y., Xia, Y., Su, J., Liu, L., Chen, K., & Peng, Q. (2024) · Lecture notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering · doi:10.1007/978-3-031-54528-3_5
MACC: A Heterogeneous Multi-Agent Mobile Edge Coding Caching Strategy for Reducing Traffic Load
Yuan, X., & Chen, N. (2023) · IEEE International Conference on Parallel and Distributed Systems (ICPADS) · doi:10.1109/icpads60453.2023.00155
Yami, R. M., Khazaei, A. A., & Rahati, S. (2024) · International Symposium on Telecommunications · doi:10.1109/ist64061.2024.10843542
Data-driven multi-criteria decision making for cell performance detection in 5G and B5G networksPreprint
Yami, R. M., Khazaei, A. A., & Rahati, S. (2025) · Research Square · doi:10.21203/rs.3.rs-5928361/v1
Implementation of Converged Gateway for Airport Heterogeneous Networks Based on Edge Cache
Su, H., Gao, Q., Li, X., & Zhang, S. (2024) · IEEE International Conference on Communication Technology (ICCT) · doi:10.1109/icct62411.2024.10946400
A low latency secure video request service scheme for mobile scenarios
Shen, H., & Guo, P. (2025) · International Conference on Communications, Information System and Computer Engineering (CISCE) · doi:10.1109/cisce65916.2025.11065615
DOJS: A Distributed Online Joint Scheme to Optimize Cost in Mobile Edge Networks
Teng, Z., Fang, J., & Xiong, N. N. (2026) · IEEE Transactions on Network and Service Management · doi:10.1109/tnsm.2026.3665360
Cooperative Edge Caching With Large Language Model in Wireless Networks
Yang, N., Wang, W. Y., Ouyang, L., & Zhang, H. (2026) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2026.3687739
Rethinking Cache Eviction: Low-Overhead and Precise Learning-based Design
Wang, Q., Xu, C., Zhou, W., Fang, J., Xiong, Y., & Niu, Z. (2026) · ACM Transactions on Computer Systems · doi:10.1145/3839232
A Survey of AI-Enabled Architectures for Next-Generation Wireless Communication Networks
Javed, A., Ahmed, I., Basit, A., Akhtar, F., Ali, H., & Khalid, M. (2026) · Wireless Personal Communications · doi:10.1007/s11277-026-12228-5
Metadata
- Authors
- Sadeghi, A., Sheikholeslami, F., & Giannakis, G. B.
- Affiliations
- University of Minnesota (US)
- Venue
- IEEE Journal of Selected Topics in Signal Processing
- Published
- 2018-02
- Volume
- 12
- Issue
- 1
- Pages
- 180-190
- ISSN
- 1932-4553 (ISSN-L)
- 1941-0484
- OpenAlex
- W2746390786
- Cited by
- 257
- References
- 39
- Citations by year
- 2026: 5 · 2025: 5 · 2024: 23 · 2023: 21 · 2022: 32 · 2021: 41 · 2020: 57 · 2019: 58 · 2018: 15
- FWCI
- 23.68
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE Journal of Selected Topics in Signal Processing (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Science Foundation (1423316, 1508993, 1514056, 1711471)
- Topics
- Physical Sciences › Computer Science › Computer Networks and Communications › Caching and Content Delivery
- Physical Sciences › Computer Science › Computer Networks and Communications › IoT and Edge/Fog Computing
- Physical Sciences › Computer Science › Computer Networks and Communications › Age of Information Optimization
- Keywords
- content caching, small cells, 5G, reinforcement learning, Q-learning, prefetching, edge caching, Markov processes, linear function approximation, scalability, heterogeneous networks
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
