The DOI matches its Crossref record; 41 of the 41 references it cites were found in a registry or an index.
Optimal and Scalable Caching for 5G Using Reinforcement Learning of Space-Time Popularities
Open accessAbstract
Small basestations (SBs) equipped with caching units have potential to handle the unprecedented demand growth in heterogeneous networks. Through low-rate, backhaul connections with the backbone, SBs can prefetch popular files during off-peak traffic hours, and service them to the edge at peak periods. To intelligently prefetch, each SB must learn what and when to cache, while taking into account SB memory limitations, the massive number of available contents, the unknown popularity profiles, as well as the space-time popularity dynamics of user file requests. In this paper, local and global Markov processes model user requests, and a reinforcement learning (RL) framework is put forth for finding the optimal caching policy when the transition probabilities involved are unknown. Joint consideration of global and local popularity demands along with cache-refreshing costs allow for a simple, yet practical asynchronous caching approach. The novel RL-based caching relies on a Q-learning algorithm to implement the optimal policy in an online fashion, thus, enabling the cache control unit at the SB to learn, track, and possibly adapt to the underlying dynamics. To endow the algorithm with scalability, a linear function approximation of the proposed Q-learning scheme is introduced, offering faster convergence as well as reduced complexity and memory requirements. Numerical tests corroborate the merits of the proposed approach in various realistic settings.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“But [7] is based on Q-Learning, which is not feasible for the practical MEC system where the stateaction space is tremendous”
From In-Edge AI: Intelligentizing Mobile Edge Computing, Caching and Communication by Federated Learning (2019) · cites this paper as [7]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. (2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 25,268
Artificial intelligence: a modern approach
Kumar, D. A., Balasubramanian, S., Byeon, D. H., & Manerkar, P. G. V. (1995) · Choice Reviews Online · doi:10.5860/choice.33-1577 · Cited by 22,191
Watkins, C., & Dayan, P. (1992) · Machine-mediated learning · doi:10.1023/a:1022676722315 · Cited by 14,277
Continuous control with deep reinforcement learning
Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., & Wierstra, D. (2015) · arXiv · doi:10.48550/arxiv.1509.02971 · Cited by 10,782
Artificial intelligence: A modern approach
Nilsson, N. J. (1996) · Artificial Intelligence · doi:10.1016/0004-3702(96)00007-0 · Cited by 10,730
Watkins, C. J. C. H., & Dayan, P. (1992) · Machine Learning · doi:10.1007/bf00992698 · Cited by 9,083
Reinforcement Learning: An Introduction
Johnson, J. D., Li, J., & Chen, Z. (2000) · Neurocomputing · doi:10.1016/s0925-2312(00)00324-6 · Cited by 8,825
Andrews, J. G., Buzzi, S., Choi, W., Hanly, S. V., Lozano, A., Soong, A. C. K., & Zhang, J. C. (2014) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2014.2328098 · Cited by 8,106
Web caching and Zipf-like distributions: evidence and implications
Breslau, L., Cao, P., Fan, L., Phillips, G., & Shenker, S. J. (1999) · IEEE Conference on Computer Communications (INFOCOM) · doi:10.1109/infcom.1999.749260 · Cited by 3,485
Device-to-device communication as an underlay to LTE-advanced networks
Doppler, K., Rinne, M., Wijting, C., Ribeiro, C. B., & Hugl, K. (2009) · IEEE Communications Magazine · doi:10.1109/mcom.2009.5350367 · Cited by 1,904
Research works printed but not linked: 1
- lillicrap, 2016, Proc Int Conf Learn Representations
Cited by
Sorted by citations
In-Edge AI: Intelligentizing Mobile Edge Computing, Caching and Communication by Federated Learning
Wang, X., Han, Y., Wang, C., Zhao, Q., Chen, X., & Chen, M. (2019) · IEEE Network · doi:10.1109/mnet.2019.1800286 · Cited by 1,030
“Particularly, related studies on edge computing and caching, such as [7] [8], have shown that reinforcement”
BackgroundMethodMorocho-Cayamcela, M. E., Lee, H., & Lim, W. (2019) · IEEE Access · doi:10.1109/access.2019.2942390 · Cited by 442
Deep Reinforcement Learning for Cooperative Content Caching in Vehicular Edge Computing and Networks
Qiao, G., Leng, S., Maharjan, S., Zhang, Y., & Ansari, N. (2019) · IEEE Internet of Things Journal · doi:10.1109/jiot.2019.2945640 · Cited by 321
The Role of Caching in Future Communication Systems and Networks
Paschos, G. S., Iosifidis, G., Tao, M., Towsley, D., & Caire, G. (2018) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2018.2844939 · Cited by 259
Online Proactive Caching in Mobile Edge Computing Using Bidirectional Deep Recurrent Neural Network
Ale, L., Zhang, N., Wu, H., Chen, D., & Han, T. (2019) · IEEE Internet of Things Journal · doi:10.1109/jiot.2019.2903245 · Cited by 187
Metadata
- Authors
- Sadeghi, A., Sheikholeslami, F., & Giannakis, G. B.
- Affiliations
- University of Minnesota (US)
- Venue
- IEEE Journal of Selected Topics in Signal Processing
- Published
- 2018-02
- Volume
- 12
- Issue
- 1
- Pages
- 180-190
- ISSN
- 1932-4553 (ISSN-L)
- 1941-0484
- OpenAlex
- W2746390786
- Cited by
- 257
- References
- 39
- Citations by year
- 2026: 5 · 2025: 5 · 2024: 23 · 2023: 21 · 2022: 32 · 2021: 41 · 2020: 57 · 2019: 58 · 2018: 15
- FWCI
- 23.68
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE Journal of Selected Topics in Signal Processing (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Science Foundation (1423316, 1508993, 1514056, 1711471)
- Topics
- Physical Sciences › Computer Science › Computer Networks and Communications › Caching and Content Delivery
- Physical Sciences › Computer Science › Computer Networks and Communications › IoT and Edge/Fog Computing
- Physical Sciences › Computer Science › Computer Networks and Communications › Age of Information Optimization
- Keywords
- content caching, small cells, 5G, reinforcement learning, Q-learning, prefetching, edge caching, Markov processes, linear function approximation, scalability, heterogeneous networks
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
