Optimal and Scalable Caching for 5G Using Reinforcement Learning of Space-Time Popularities
Open accessAbstract
Small basestations (SBs) equipped with caching units have potential to handle the unprecedented demand growth in heterogeneous networks. Through low-rate, backhaul connections with the backbone, SBs can prefetch popular files during off-peak traffic hours, and service them to the edge at peak periods. To intelligently prefetch, each SB must learn what and when to cache, while taking into account SB memory limitations, the massive number of available contents, the unknown popularity profiles, as well as the space-time popularity dynamics of user file requests. In this paper, local and global Markov processes model user requests, and a reinforcement learning (RL) framework is put forth for finding the optimal caching policy when the transition probabilities involved are unknown. Joint consideration of global and local popularity demands along with cache-refreshing costs allow for a simple, yet practical asynchronous caching approach. The novel RL-based caching relies on a Q-learning algorithm to implement the optimal policy in an online fashion, thus, enabling the cache control unit at the SB to learn, track, and possibly adapt to the underlying dynamics. To endow the algorithm with scalability, a linear function approximation of the proposed Q-learning scheme is introduced, offering faster convergence as well as reduced complexity and memory requirements. Numerical tests corroborate the merits of the proposed approach in various realistic settings.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
[7] is based on Q-Learning, which is not feasible for a practical MEC system where the state-action space is tremendous.
“But [7] is based on Q-Learning, which is not feasible for the practical MEC system where the stateaction space is tremendous”
From In-Edge AI: Intelligentizing Mobile Edge Computing, Caching and Communication by Federated Learning (2019) · cites this paper as [7]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. (2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 25,263 · see citing works
Artificial intelligence: a modern approach
Kumar, D. A., Balasubramanian, S., Byeon, D. H., & Manerkar, P. G. V. (1995) · Choice Reviews Online · doi:10.5860/choice.33-1577 · Cited by 22,182 · see citing works
Watkins, C., & Dayan, P. (1992) · Machine-mediated learning · doi:10.1023/a:1022676722315 · Cited by 14,277 · see citing works
Continuous control with deep reinforcement learning
Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., & Wierstra, D. (2015) · arXiv · doi:10.48550/arxiv.1509.02971 · Cited by 10,782 · see citing works
Artificial intelligence: A modern approach
Nilsson, N. J. (1996) · Artificial Intelligence · doi:10.1016/0004-3702(96)00007-0 · Cited by 10,726 · see citing works
Reinforcement Learning: An Introduction
Johnson, J. D., Li, J., & Chen, Z. (2000) · Neurocomputing · doi:10.1016/s0925-2312(00)00324-6 · Cited by 8,821 · see citing works
Andrews, J. G., Buzzi, S., Choi, W., Hanly, S. V., Lozano, A., Soong, A. C. K., & Zhang, J. C. (2014) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2014.2328098 · Cited by 8,106 · see citing works
Web caching and Zipf-like distributions: evidence and implications
Breslau, L., Cao, P., Fan, L., Phillips, G., & Shenker, S. J. (1999) · IEEE Conference on Computer Communications (INFOCOM) · doi:10.1109/infcom.1999.749260 · Cited by 3,485 · see citing works
Device-to-device communication as an underlay to LTE-advanced networks
Doppler, K., Rinne, M., Wijting, C., Ribeiro, C. B., & Hugl, K. (2009) · IEEE Communications Magazine · doi:10.1109/mcom.2009.5350367 · Cited by 1,904 · see citing works
Maddah-Ali, M. A., & Niesen, U. (2014) · IEEE Transactions on Information Theory · doi:10.1109/tit.2014.2306938 · Cited by 1,726 · see citing works
Femtocells: Past, Present, and Future
Andrews, J. G., Claußen, H., Döhler, M., Rangan, S., & Reed, M. C. (2012) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2012.120401 · Cited by 1,184 · see citing works
Cache in the air: exploiting content caching and delivery techniques for 5G systems
Wang, X., Chen, M., Taleb, T., Ksentini, A., & Leung, V. (2014) · IEEE Communications Magazine · doi:10.1109/mcom.2014.6736753 · Cited by 1,063 · cited 1× · see citing works
“, reusable video streams, during off-peak periods, storing this data in SBs equipped with memory units, and reusing them during peak traffic hours [2]–[4].”
MethodFemtocaching and device-to-device collaboration: A new architecture for wireless video distribution
Golrezaei, N., Molisch, A. F., Dimakis, A. G., & Caire, G. (2013) · IEEE Communications Magazine · doi:10.1109/mcom.2013.6495773 · Cited by 686 · see citing works
Decentralized Coded Caching Attains Order-Optimal Memory-Rate Tradeoff
Maddah-Ali, M. A., & Niesen, U. (2014) · IEEE/ACM Transactions on Networking · doi:10.1109/tnet.2014.2317316 · Cited by 667 · see citing works
Fundamental Limits of Caching in Wireless D2D Networks
Ji, M., Caire, G., & Molisch, A. F. (2015) · IEEE Transactions on Information Theory · doi:10.1109/tit.2015.2504556 · Cited by 635 · see citing works
The O.D.E. Method for Convergence of Stochastic Approximation and Reinforcement Learning
Borkar, V. S., & Meyn, S. (2000) · SIAM Journal on Control and Optimization · doi:10.1137/s0363012997331639 · Cited by 528 · see citing works
Wireless Device-to-Device Caching Networks: Basic Principles and System Performance
Ji, M., Caire, G., & Molisch, A. F. (2015) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2015.2452672 · Cited by 445 · see citing works
Optimal geographic caching in cellular networks
Błaszczyszyn, B., & Giovanidis, A. (2015) · International Conference on Communications · doi:10.1109/icc.2015.7248843 · Cited by 440 · see citing works
Kernel-Based Reinforcement Learning
Ormoneit, D., & Sen, S. (2017) · Encyclopedia of Machine Learning and Data Mining · doi:10.1007/978-1-4899-7687-1_100235 · Cited by 381 · see citing works
Wireless caching: technical misconceptions and business barriers
Paschos, G. S., Baştuǧ, E., Land, I., Caire, G., & Debbah, M. A. (2016) · IEEE Communications Magazine · doi:10.1109/mcom.2016.7537172 · Cited by 321 · see citing works
Context-Aware Proactive Content Caching With Service Differentiation in Wireless Networks
Muller, S., Atan, O., van der Schaar, M., & Klein, A. (2016) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2016.2636139 · Cited by 282 · see citing works
Kernel-Based Reinforcement Learning
Ormoneit, D., & Sen, Ś. (2002) · Machine Learning · doi:10.1023/a:1017928328829 · Cited by 254 · see citing works
Pedarsani, R., Maddah-Ali, M. A., & Niesen, U. (2015) · IEEE/ACM Transactions on Networking · doi:10.1109/tnet.2015.2394482 · Cited by 236 · see citing works
A Learning-Based Approach to Caching in Heterogenous Small Cell Networks
Bharath, B. N., Nagananda, K. G., & Poor, H. V. (2016) · IEEE Transactions on Communications · doi:10.1109/tcomm.2016.2536728 · Cited by 202 · see citing works
Placing dynamic content in caches with small population
Leconte, M., Paschos, G. S., Gkatzikis, L., Draief, M., Vassilaras, S., & Chouvardas, S. (2016) · IEEE Conference on Computer Communications (INFOCOM) · doi:10.1109/infocom.2016.7524380 · Cited by 155 · see citing works
Learning distributed caching strategies in small cell networks
Sengupta, A., Amuru, S., Tandon, R., Buehrer, R. M., & Clancy, T. C. (2014) · International Symposium on Wireless Communication Systems · doi:10.1109/iswcs.2014.6933484 · Cited by 141 · see citing works
A Tutorial on Linear Function Approximators for Dynamic Programming and Reinforcement Learning
Geramifard, A., Walsh, T. J., Tellex, S., Chowdhary, G. V., Roy, N., & How, J. P. (2013) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000042 · Cited by 124 · see citing works
Video delivery over heterogeneous cellular networks: Optimizing cost and performance
Poularakis, K., Iosifidis, G., Argyriou, A., & Tassiulas, L. (2014) · IEEE Conference on Computer Communications (INFOCOM) · doi:10.1109/infocom.2014.6848038 · Cited by 112 · see citing works
Trend-Aware Video Caching Through Online Learning
Li, S., Xu, J., van der Schaar, M., & Li, W. (2016) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2016.2596042 · Cited by 98 · see citing works
Decentralized coded caching attains order-optimal memory-rate tradeoff
Maddah-Ali, M. A., & Niesen, U. (2013) · Allerton Conference on Communication, Control, and Computing · doi:10.1109/allerton.2013.6736555 · Cited by 95 · see citing works
A transfer learning approach for cache-enabled wireless networks
Bastug, E., Bennis, M., & Debbah, M. A. (2015) · International Symposium on Modeling and Optimization in Mobile, Ad Hoc, and Wireless Networks (WiOpt) · doi:10.1109/wiopt.2015.7151068 · Cited by 94 · see citing works
Optimal Caching and Scheduling for Cache-Enabled D2D Communications
Chen, B., Yang, C., & Xiong, Z. (2017) · IEEE Communications Letters · doi:10.1109/lcomm.2017.2652440 · Cited by 82 · see citing works
Exploration and Inference in Learning from Reinforcement
Wyatt, J. (1998) · Cited by 74 · see citing works
Content caching at the wireless network edge: A distributed algorithm via belief propagation
Liu, J., Bai, B. B., Zhang, J., & Letaief, K. B. (2016) · International Conference on Communications · doi:10.1109/icc.2016.7510807 · Cited by 61 · see citing works
Learning Representation and Control in Markov Decision Processes: New Frontiers
Sridhar, M. (2009) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000003 · Cited by 58 · see citing works
Ultra-dense edge caching under spatio-temporal demand and network dynamics
Kim, H., Park, J., Bennis, M., Kim, S., & Debbah, M. A. (2017) · International Conference on Communications · doi:10.1109/icc.2017.7996764 · Cited by 25 · see citing works
A Tutorial on Linear Function Approximators for Dynamic Programming and Reinforcement Learning
Geramifard, A. (2013) · now publishers, Inc. eBooks · doi:10.1561/9781601987617 · Cited by 19 · see citing works
Learning Representation and Control in Markov Decision Processes: New Frontiers
Mahadevan, S. (2007) · now publishers, Inc. eBooks · doi:10.1561/9781601982391 · Cited by 17 · see citing works
Blasco, P., & Gündüz, D. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1407.6154 · Cited by 15 · see citing works
Ultra-Dense Edge Caching under Spatio-Temporal Demand and Network DynamicsPreprint
Kim, H., Park, J., Bennis, M., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.01038 · Cited by 1 · see citing works
Flagged first, then by citations
Research works printed but not linked: 2
- [26][26] C. J. Watkins and P. Dayan, 'Q-learning,' Machine learning , vol. 8, no. 3-4, pp. 279-292, 1992. -
- [27][27] V. S. Borkar and S. P. Meyn, 'The ODE method for convergence of stochastic approximation and reinforcement learning,' SIAM Journal on Control and Optimization , vol. 38, no. 2, pp. 447-469, 2000. -
Cited by
Sorted by citations
In-Edge AI: Intelligentizing Mobile Edge Computing, Caching and Communication by Federated Learning
Wang, X., Han, Y., Wang, C., Zhao, Q., Chen, X., & Chen, M. (2019) · IEEE Network · doi:10.1109/mnet.2019.1800286 · Cited by 1,030 · see citing works
“Particularly, related studies on edge computing and caching, such as [7] [8], have shown that reinforcement”
BackgroundMethodMorocho-Cayamcela, M. E., Lee, H., & Lim, W. (2019) · IEEE Access · doi:10.1109/access.2019.2942390 · Cited by 418 · see citing works
Deep Reinforcement Learning for Cooperative Content Caching in Vehicular Edge Computing and Networks
Qiao, G., Leng, S., Maharjan, S., Zhang, Y., & Ansari, N. (2019) · IEEE Internet of Things Journal · doi:10.1109/jiot.2019.2945640 · Cited by 321 · see citing works
The Role of Caching in Future Communication Systems and Networks
Paschos, G. S., Iosifidis, G., Tao, M., Towsley, D., & Caire, G. (2018) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2018.2844939 · Cited by 259 · see citing works
Online Proactive Caching in Mobile Edge Computing Using Bidirectional Deep Recurrent Neural Network
Ale, L., Zhang, N., Wu, H., Chen, D., & Han, T. (2019) · IEEE Internet of Things Journal · doi:10.1109/jiot.2019.2903245 · Cited by 186 · see citing works
Elgendy, I. A., Zhang, W.-Z., He, H. X., Gupta, B. B., & El‐Latif, A. A. A. (2021) · Wireless Networks · doi:10.1007/s11276-021-02554-w · Cited by 169 · see citing works
Al-Hilo, A., Samir, M., Assi, C. M., Sharafeddine, S., & Ebrahimi, D. (2020) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2020.3020220 · Cited by 166 · see citing works
A survey of 5G network systems: challenges and machine learning approaches
Fourati, H., Mâaloul, R., & Fourati, L. C. (2020) · International Journal of Machine Learning and Cybernetics · doi:10.1007/s13042-020-01178-4 · Cited by 166 · see citing works
Deep Reinforcement Learning-Based Edge Caching in Wireless Networks
Zhong, C., Gursoy, M. C., & Velipasalar, S. (2020) · IEEE Transactions on Cognitive Communications and Networking · doi:10.1109/tccn.2020.2968326 · Cited by 159 · see citing works
Shuja, J., Bilal, K., Alasmary, W., Sinky, H., & Alanazi, E. (2021) · Journal of Network and Computer Applications · doi:10.1016/j.jnca.2021.103005 · Cited by 157 · see citing works
Narayanan, A., Verma, S. M., Ramadan, E., Babaie, P., & Zhang, Z.-L. (2018) · ACM SIGCOMM Workshop on Network Meets AI & ML (NetAI) · doi:10.1145/3229543.3229555 · Cited by 147 · see citing works
Deep Reinforcement Learning for Adaptive Caching in Hierarchical Content Delivery Networks
Sadeghi, A., Wang, G., & Giannakis, G. B. (2019) · IEEE Transactions on Cognitive Communications and Networking · doi:10.1109/tccn.2019.2936193 · Cited by 125 · see citing works
Self-citationAI-Based Resource Provisioning of IoE Services in 6G: A Deep Reinforcement Learning Approach
Sami, H., Otrok, H., Bentahar, J., & Mourad, A. (2021) · IEEE Transactions on Network and Service Management · doi:10.1109/tnsm.2021.3066625 · Cited by 114 · see citing works
Liu, X., Zhang, H., Long, K., Zhou, M., Li, Y., & Poor, H. V. (2022) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2022.3158696 · Cited by 112 · see citing works
Deep Reinforcement Learning-Enabled Secure Visible Light Communication Against Eavesdropping
Xiao, L., Sheng, G., Liu, S., Dai, H., Peng, M., & Song, J. (2019) · IEEE Transactions on Communications · doi:10.1109/tcomm.2019.2930247 · Cited by 109 · see citing works
A Survey on Securing Federated Learning: Analysis of Applications, Attacks, Challenges, and Trends
Neto, H. N. C., Hribar, J., Dusparić, I., Mattos, D. M. F., & Fernandes, N. C. (2023) · IEEE Access · doi:10.1109/access.2023.3269980 · Cited by 108 · see citing works
Demand-Driven Deep Reinforcement Learning for Scalable Fog and Service Placement
Sami, H., Mourad, A., Otrok, H., & Bentahar, J. (2021) · IEEE Transactions on Services Computing · doi:10.1109/tsc.2021.3075988 · Cited by 98 · see citing works
Learning and Management for Internet of Things: Accounting for Adaptivity and Scalability
Chen, T., Barbarossa, S., Wang, X., Giannakis, G. B., & Zhang, Z.-L. (2019) · Proceedings of the IEEE · doi:10.1109/jproc.2019.2896243 · Cited by 93 · see citing works
Self-citationOnline Content Popularity Prediction and Learning in Wireless Edge Caching
Garg, N., Sellathurai, M., Bhatia, V., Bharath, B. N., & Ratnarajah, T. (2020) · IEEE Transactions on Communications · doi:10.1109/tcomm.2019.2956041 · Cited by 89 · see citing works
Jiang, X., Yu, F. R., Song, T., & Leung, V. C. M. (2022) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2021.3065209 · Cited by 88 · see citing works
Enabling technologies for AI empowered 6G massive radio access networks
Shahjalal, M. A., Kim, W., Khalid, W., Moon, S., Khan, M. A., Liu, S., Lim, S., Kim, E., Yun, D.-W., Lee, J., et al. (2022) · ICT Express · doi:10.1016/j.icte.2022.07.002 · Cited by 92 · see citing works
Learning to Cache With No Regrets
Paschos, G. S., Destounis, A., Vigneri, L., & Iosifidis, G. (2019) · IEEE Conference on Computer Communications (INFOCOM) · doi:10.1109/infocom.2019.8737446 · Cited by 85 · see citing works
Integrating Edge Intelligence and Blockchain: What, Why, and How
Wang, X., Ren, X., Qiu, C., Xiong, Z., Yao, H., & Leung, V. C. M. (2022) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2022.3189962 · Cited by 83 · see citing works
Distributed Reinforcement Learning for Privacy-Preserving Dynamic Edge Caching
Liu, S., Zheng, C., Huang, Y., & Quek, T. Q. S. (2022) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2022.3142348 · Cited by 80 · see citing works
Caching Transient Content for IoT Sensing: Multi-Agent Soft Actor-Critic
Wu, X., Li, X., Li, J., Ching, P. C., Leung, V. C. M., & Poor, H. V. (2021) · IEEE Transactions on Communications · doi:10.1109/tcomm.2021.3086535 · Cited by 77 · see citing works
Tang, J., Tang, H., Zhang, X. Y., Cumanan, K., Chen, G., Wong, K., & Chambers, J. A. (2019) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2019.2954127 · Cited by 67 · see citing works
Artificial Intelligence for Wireless Caching: Schemes, Performance, and Challenges
Sheraz, M., Ahmed, M., Hou, X., Li, Y., Jin, D., Han, Z., & Jiang, T. (2020) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2020.3008362 · Cited by 63 · see citing works
Chen, S., Yao, Z., Jiang, X., Yang, J., & Hanzo, L. (2020) · IEEE Transactions on Communications · doi:10.1109/tcomm.2020.3044298 · Cited by 63 · see citing works
Ensemble reinforcement learning: A survey
Song, Y., Suganthan, P. N., Pedrycz, W., Ou, J., He, Y., Chen, Y., & Wu, Y. (2023) · Applied Soft Computing · doi:10.1016/j.asoc.2023.110975 · Cited by 64 · see citing works
Reinforcement Learning for Adaptive Caching With Dynamic Storage Pricing
Sadeghi, A., Sheikholeslami, F., Marques, A. G., & Giannakis, G. B. (2019) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2019.2933780 · Cited by 60 · see citing works
Self-citationLearning Automata Based Q-Learning for Content Placement in Cooperative Caching
Yang, Z., Liu, Y., Chen, Y., & Jiao, L. (2020) · IEEE Transactions on Communications · doi:10.1109/tcomm.2020.2982136 · Cited by 59 · see citing works
Energy-Efficient Proactive Caching for Adaptive Video Streaming via Data-Driven Optimization
Li, L., Shi, D., Hou, R., Chen, R., Lin, B., & Pan, M. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.2981250 · Cited by 59 · see citing works
Collaborative Multi-Agent Multi-Armed Bandit Learning for Small-Cell Caching
Xu, X., Tao, M., & Shen, C. (2020) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2020.2966599 · Cited by 57 · see citing works
Joint task offloading and data caching in mobile edge computing networks
Zhang, N., Guo, S., Dong, Y., & Liu, D. (2020) · Computer Networks · doi:10.1016/j.comnet.2020.107446 · Cited by 55 · see citing works
MARL-Based Distributed Cache Placement for Wireless Networks
Lin, X., Tang, Y., Lei, X., Xia, J., Zhou, Q., Wu, H., & Fan, L. (2019) · IEEE Access · doi:10.1109/access.2019.2916155 · Cited by 52 · see citing works
Using Grouped Linear Prediction and Accelerated Reinforcement Learning for Online Content Caching
Zhang, N., Zheng, K., & Tao, M. (2018) · IEEE International Conference on Communications workshops · doi:10.1109/iccw.2018.8403712 · Cited by 51 · see citing works
Device-to-device content caching techniques in 5G: A taxonomy, solutions, and challenges
Prerna, D., Tekchandani, R., & Kumar, N. (2020) · Computer Communications · doi:10.1016/j.comcom.2020.01.057 · Cited by 50 · see citing works
A Mean Field Game-Based Distributed Edge Caching in Fog Radio Access Networks
Jiang, Y., Hu, Y., Bennis, M., Zheng, F., & You, X. (2019) · IEEE Transactions on Communications · doi:10.1109/tcomm.2019.2961081 · Cited by 49 · see citing works
Cocktail Edge Caching: Ride Dynamic Trends of Content Popularity With Ensemble Learning
Zong, T., Li, C., Lei, Y., Li, G., Cao, H., & Liu, Y. (2022) · IEEE/ACM Transactions on Networking · doi:10.1109/tnet.2022.3193680 · Cited by 47 · see citing works
Shan, N., Cui, X., & Gao, Z. (2020) · Computer Communications · doi:10.1016/j.comcom.2020.05.037 · Cited by 49 · see citing works
Orchestration and Control in Software‐Defined 5G Networks: Research Challenges
Nencioni, G., Garroppo, R. G., Gonzalez, A. J., Helvik, B. E., & Procissi, G. (2018) · Wireless Communications and Mobile Computing · doi:10.1155/2018/6923867 · Cited by 44 · see citing works
Multi-Agent Reinforcement Learning for Cooperative Coded Caching via Homotopy Optimization
Wu, X., Li, J., Xiao, M., Ching, P. C., & Poor, H. V. (2021) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2021.3066458 · Cited by 44 · see citing works
Hybrid Policy Learning for Energy-Latency Tradeoff in MEC-Assisted VR Video Service
Zheng, C., Liu, S., Huang, Y., & Yang, L. (2021) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2021.3099129 · Cited by 44 · see citing works
Towards Understanding Asynchronous Advantage Actor-Critic: Convergence and Linear Speedup
Shen, H., Zhang, K., Hong, M., & Chen, T. (2023) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2023.3268475 · Cited by 43 · see citing works
Yan, Y., Zhang, B., Li, C., & Su, C. (2020) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2020.3042089 · Cited by 42 · see citing works
Reinforcement Learning Meets Wireless Networks: A Layering Perspective
Chen, Y., Liu, Y., Zeng, M., Saleem, U., Lu, Z., Wen, X., Jin, D., Han, Z., Jiang, T., & Li, Y. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.3025365 · Cited by 40 · see citing works
Predictive Proactive Caching in VANETs for Social Networking
Elsayed, S. A., Abdelhamid, S., & Hassanein, H. S. (2022) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2022.3148720 · Cited by 37 · see citing works
Zheng, C., Liu, S., Huang, Y., Zhang, W., & Yang, L. (2022) · IEEE Internet of Things Journal · doi:10.1109/jiot.2022.3189055 · Cited by 37 · see citing works
Digital Twin-Assisted Data-Driven Optimization for Reliable Edge Caching in Wireless Networks
Zhang, Z., Liu, Y., Peng, Z., Chen, M., Xu, D., & Cui, S. R. (2024) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2024.3431575 · Cited by 35 · see citing works
Almutairi, M. S. (2022) · Mathematical Problems in Engineering · doi:10.1155/2022/6855435 · Cited by 34 · see citing works
Reinforcement learning algorithm for 5G indoor device‐to‐device communications
Sreedevi, A. G., & Rao, T. R. (2019) · Transactions on Emerging Telecommunications Technologies · doi:10.1002/ett.3670 · Cited by 33 · see citing works
Online Convex Optimization for Caching Networks
Paschos, G. S., Destounis, A., & Iosifidis, G. (2020) · IEEE/ACM Transactions on Networking · doi:10.1109/tnet.2020.2968424 · Cited by 33 · see citing works
FScaler: Automatic Resource Scaling of Containers in Fog Clusters Using Reinforcement Learning
Sami, H., Mourad, A., Otrok, H., & Bentahar, J. (2020) · International Wireless Communications and Mobile Computing Conference (IWCMC) · doi:10.1109/iwcmc48107.2020.9148401 · Cited by 33 · see citing works
Gao, S., Dong, P., Pan, Z., & Li, G. Y. (2020) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2020.2979918 · Cited by 34 · see citing works
A Reinforcement Learning Based Smart Cache Strategy for Cache-Aided Ultra-Dense Network
Li, W., Wang, J., Zhang, G., Li, L., Dang, Z., & Li, S. (2019) · IEEE Access · doi:10.1109/access.2019.2905589 · Cited by 30 · see citing works
Guo, B., Zhang, X., Sheng, Q., & Yang, H. (2020) · IEEE Access · doi:10.1109/access.2020.2964258 · Cited by 31 · see citing works
AutoSight: Distributed Edge Caching in Short Video Network
Zhang, Y., Li, P., Zhang, Z.-L., Bai, B. B., Zhang, G., Wang, W., Lian, B., & Xu, K. (2020) · IEEE Network · doi:10.1109/mnet.001.1900345 · Cited by 30 · see citing works
Somesula, M. K., Rout, R. R., & Somayajulu, D. V. L. N. (2022) · Computer Networks · doi:10.1016/j.comnet.2022.108876 · Cited by 30 · see citing works
Distributed Edge Caching via Reinforcement Learning in Fog Radio Access Networks
Lu, L., Jiang, Y., Bennis, M., Ding, Z., Zheng, F., & You, X. (2019) · IEEE Vehicular Technology Conference · doi:10.1109/vtcspring.2019.8746321 · Cited by 29 · see citing works
Jazaeri, S. S., Asghari, P., Jabbehdari, S., & Javadi, H. H. S. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-023-15657-7 · Cited by 29 · see citing works
Joint Caching and Routing in Congestible Networks of Arbitrary Topology
Liu, B., Poularakis, K., Tassiulas, L., & Jiang, T. (2019) · IEEE Internet of Things Journal · doi:10.1109/jiot.2019.2935742 · Cited by 28 · see citing works
Information fusion on delivery: A survey on the roles of mobile edge caching systems
Nguyen, T.-V., Tran, A.-T., Dao, N., Moon, H., & Cho, S. (2022) · Information Fusion · doi:10.1016/j.inffus.2022.08.029 · Cited by 28 · see citing works
Learning-Based Hierarchical Edge Caching for Cloud-Aided Heterogeneous Networks
Li, D., Zhang, H., Yuan, D., & Zhang, M. (2022) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2022.3206236 · Cited by 28 · see citing works
A View on Deep Reinforcement Learning in System OptimizationPreprint
Haj-Ali, A., Ahmed, N. K., Willke, T. L., Gonzalez, J. E., Asanović, K., & Stoica, I. G. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.01275 · Cited by 39 · see citing works
Agile Cache Replacement in Edge Computing via Offline-Online Deep Reinforcement Learning
Wang, Z., Hu, J. W., Min, G., Zhao, Z., & Wang, Z. (2024) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2024.3368763 · Cited by 27 · see citing works
Caching in Dynamic Environments: A Near-Optimal Online Learning Approach
Zhou, S., Wang, Z., Hu, C., Mao, Y., Yan, H., Zhang, S., Wu, C., & Zhu, W. (2021) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2021.3132156 · Cited by 26 · see citing works
Cache Optimization Models and Algorithms
Paschos, G. S., Iosifidis, G., & Caire, G. (2020) · Foundations and Trends® in Communications and Information Theory · doi:10.1561/0100000104 · Cited by 9 · see citing works
An Adaptive Cooperative Caching Strategy for Vehicular Networks
Jin, Z., Song, T., & Jia, W. (2024) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2024.3367543 · Cited by 26 · see citing works
Joint Long-Term Cache Updating and Short-Term Content Delivery in Cloud-Based Small Cell Networks
Wu, X., Li, Q., Li, X., Leung, V. C. M., & Ching, P. C. (2020) · IEEE Transactions on Communications · doi:10.1109/tcomm.2020.2970986 · Cited by 26 · see citing works
Chuang, Y.-C., Chiu, W., Chang, R. Y., & Lai, Y. (2023) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2023.3259109 · Cited by 25 · see citing works
Zhou, X., Liu, Z., Guo, M., Zhao, J., & Wang, J. (2021) · IEEE Transactions on Emerging Topics in Computing · doi:10.1109/tetc.2021.3115793 · Cited by 25 · see citing works
Proactive Edge Caching in Content-Centric Networks With Massive Dynamic Content Requests
Xu, X., Feng, C., Shan, S., Zhang, T., & Loo, J. K. K. (2020) · IEEE Access · doi:10.1109/access.2020.2983068 · Cited by 24 · see citing works
The Edge Cloud: A Holistic View of Communication, Computation, and Caching
Barbarossa, S., Sardellitti, S., Ceci, E., & Merluzzi, M. (2018) · Elsevier eBooks · doi:10.1016/b978-0-12-813677-5.00016-x · Cited by 24 · see citing works
Teng, Z., Fang, J., & Liu, Y. (2024) · IEEE Transactions on Network and Service Management · doi:10.1109/tnsm.2024.3361796 · Cited by 23 · see citing works
A Review on Machine Learning for Channel Coding
Kee, H. L. M., Ahmad, N. B., Izhar, M. A. M., Anwar, K., & Ng, S. X. (2024) · IEEE Access · doi:10.1109/access.2024.3412192 · Cited by 22 · see citing works
Chang, R. Y., Han, S.-F., & Chien, F. (2021) · IEEE Vehicular Technology Conference · doi:10.1109/vtc2021-fall52928.2021.9625449 · Cited by 22 · see citing works
Making content caching policies 'smart' using the deepcache framework
Narayanan, A., Verma, S., Ramadan, E., Babaie, P., & Zhang, Z.-L. (2019) · ACM SIGCOMM Computer Communication Review · doi:10.1145/3310165.3310174 · Cited by 22 · see citing works
The Study of Dynamic Caching via State Transition Field—the Case of Time-Invariant Popularity
Gao, J., Zhao, L. Y., & She, X. (2019) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2019.2940676 · Cited by 22 · see citing works
Optimal caching policy for wireless content delivery in D2D networks
Ali, A. S., Mahmoud, K. R., & Naguib, K. M. (2019) · Journal of Network and Computer Applications · doi:10.1016/j.jnca.2019.102467 · Cited by 22 · see citing works
Dynamic Caching Content Replacement in Base Station Assisted Wireless D2D Caching Networks
Lee, M., Feng, H., & Molisch, A. F. (2020) · IEEE Access · doi:10.1109/access.2020.2973953 · Cited by 22 · see citing works
Machine-Learning Approach for User Association and Content Placement in Fog Radio Access Networks
Yan, S., Jiao, M., Zhou, Y., Peng, M., & Daneshmand, M. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.2973339 · Cited by 22 · see citing works
Intelligent Content Precaching Scheme for Platoon-Based Edge Vehicular Networks
Wu, Y., Fang, X., Luo, C., & Min, G. (2022) · IEEE Internet of Things Journal · doi:10.1109/jiot.2022.3178099 · Cited by 21 · see citing works
On the Opportunities and Challenges of NOMA-Based Fog Radio Access Networks: An Overview
Kaneko, M., Randrianantenaina, I., Dahrouj, H., ElSawy, H., & Alouini, M. (2020) · IEEE Access · doi:10.1109/access.2020.3037183 · Cited by 21 · see citing works
Q-Learning-Based Spectrum Access for Content Delivery in Mobile Networks
Su, Z., Dai, M., Xu, Q., Li, R., & Fu, S. (2020) · IEEE Transactions on Cognitive Communications and Networking · doi:10.1109/tccn.2020.2971688 · Cited by 20 · see citing works
Sun, Y., Chen, S., Wang, Z., & Mao, S. (2022) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2022.3214075 · Cited by 20 · see citing works
An overview of analysis methods and evaluation results for caching strategies
Haßlinger, G., Okhovatzadeh, M., Ntougias, K., Hasslinger, F., & Hohlfeld, O. (2023) · Computer Networks · doi:10.1016/j.comnet.2023.109583 · Cited by 20 · see citing works
Learning paradigms for communication and computing technologies in IoT systems
Ejaz, W., Basharat, M., Saadat, S., Khattak, A. M., Naeem, M., & Anpalagan, A. (2020) · Computer Communications · doi:10.1016/j.comcom.2020.01.043 · Cited by 20 · see citing works
Active Content Popularity Learning and Caching Optimization With Hit Ratio Guarantees
Bommaraveni, S., Vu, T. X., Chatzinotas, S., & Ottersten, B. (2020) · IEEE Access · doi:10.1109/access.2020.3014379 · Cited by 20 · see citing works
Decentralized Multi-Agent Multi-Armed Bandit Learning With Calibration for Multi-Cell Caching
Xu, X., & Tao, M. (2020) · IEEE Transactions on Communications · doi:10.1109/tcomm.2020.3045050 · Cited by 20 · see citing works
Deep Regression Model for Videos Popularity Prediction in Mobile Edge Caching Networks
Masood, A., Nguyen, T. V., & Cho, S. (2021) · International Conference on Information Networking · doi:10.1109/icoin50884.2021.9333920 · Cited by 20 · see citing works
Yan, S., Qi, L., Zhou, Y., Peng, M., & Rahman, G. (2019) · IEEE Transactions on Communications · doi:10.1109/tcomm.2019.2950215 · Cited by 19 · see citing works
Ye, C., Cui, Y., Yang, Y., & Wang, R. (2019) · IEEE Transactions on Communications · doi:10.1109/tcomm.2019.2919724 · Cited by 19 · see citing works
Online Caching with Optimistic Learning
Mhaisen, N., Iosifidis, G., & Leith, D. J. (2022) · IFIP Networking Conference (IFIP Networking) · doi:10.23919/ifipnetworking55013.2022.9829806 · Cited by 18 · see citing works
Hu, Z., Zhang, Y., Huang, H., Wen, X., Agbodike, O., & Chen, J. (2023) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2023.105930 · Cited by 18 · see citing works
Resources Sharing in 5G Networks: Learning-Enabled Incentives and Coalitional Games
Yu, L. C., Li, Z., Liu, J., & Zhou, R. (2019) · IEEE Systems Journal · doi:10.1109/jsyst.2019.2958890 · Cited by 18 · see citing works
Function Approximation Based Reinforcement Learning for Edge Caching in Massive MIMO Networks
Garg, N., Sellathurai, M., Bhatia, V., & Ratnarajah, T. (2020) · IEEE Transactions on Communications · doi:10.1109/tcomm.2020.3047658 · Cited by 18 · see citing works
Stochastic Game Based Cooperative Alternating Q-Learning Caching in Dynamic D2D Networks
Zhang, T., Fang, X., Wang, Z., Liu, Y., & Nallanathan, A. (2021) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2021.3120292 · Cited by 18 · see citing works
The Study of Dynamic Caching via State Transition Field—the Case of Time-Varying Popularity
Gao, J., Zhao, L. Y., & Shen, X. (2019) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2019.2940667 · Cited by 17 · see citing works
Yan, J., Zhang, M., Jiang, Y., Zheng, F., Chang, Q., Abualnaja, K. M., Mumtaz, S., & You, X. (2023) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2023.3328554 · Cited by 17 · see citing works
Deep Reinforcement Learning-Based Joint Caching and Routing in AI-Driven Networks
Yang, M., Deyun Gao, A., Zhang, W., Yang, D., Niyato, D. T., Zhang, H., & Leung, V. C. M. (2024) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2024.3481276 · Cited by 17 · see citing works
- Authors
- Sadeghi, A., Sheikholeslami, F., & Giannakis, G. B.
- Affiliations
- University of Minnesota (US)
- Venue
- IEEE Journal of Selected Topics in Signal Processing
- Published
- 2018-02
- Volume
- 12
- Issue
- 1
- Pages
- 180-190
- ISSN
- 1932-4553 (ISSN-L)
- 1941-0484
- OpenAlex
- W2746390786
- Cited by
- 257
- References
- 39
- Citations by year
- 2026: 5 · 2025: 5 · 2024: 23 · 2023: 21 · 2022: 32 · 2021: 41 · 2020: 57 · 2019: 58 · 2018: 15
- FWCI
- 23.68
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE Journal of Selected Topics in Signal Processing (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Science Foundation (1423316, 1508993, 1514056, 1711471)
- Topics
- Physical Sciences › Computer Science › Computer Networks and Communications › Caching and Content Delivery
- Physical Sciences › Computer Science › Computer Networks and Communications › IoT and Edge/Fog Computing
- Physical Sciences › Computer Science › Computer Networks and Communications › Age of Information Optimization
- Keywords
- content caching, small cells, 5G, reinforcement learning, Q-learning, prefetching, edge caching, Markov processes, linear function approximation, scalability, heterogeneous networks
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
