A deep reinforcement learning-based framework for content caching
Open accessThe DOI matches its Crossref record; 16 of the 16 references it cites were found in a registry or an index.
Abstract
Content caching at the edge nodes is a promising technique to reduce the data traffic in next-generation wireless networks. Inspired by the success of Deep Reinforcement Learning (DRL) in solving complicated control problems, this work presents a DRL-based framework with Wolpertinger architecture for content caching at the base station. The proposed framework is aimed at maximizing the long-term cache hit rate, and it requires no knowledge of the content popularity distribution. To evaluate the proposed framework, we compare the performance with other caching algorithms, including Least Recently Used (LRU), Least Frequently Used (LFU), and First-In First-Out (FIFO) caching strategies. Meanwhile, since the Wolpertinger architecture can effectively limit the action space size, we also compare the performance with Deep Q-Network to identify the impact of dropping a portion of the actions. Our results show that the proposed framework can achieve improved short-term cache hit rate and improved and stable long-term cache hit rate in comparison with LRU, LFU, and FIFO schemes. Additionally, the performance is shown to be competitive in comparison to Deep Q-learning, while the proposed framework can provide significant savings in runtime.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“As illustrated in [233] [234], the complexity of optimization algorithms can be transferred to the training of DNNs, and thus breaking the practical limitation of employing them.”
From Convergence of Edge Computing and Deep Learning: A Comprehensive Survey (2020) · cites this paper as [234]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Compared DQL-based algorithms searching the whole action space, the trained DRL agent with DDPG and Wolpertinger architecture is able to achieve competitive cache hit rates while reducing the runtime.”
From Convergence of Edge Computing and Deep Learning: A Comprehensive Survey (2020) · cites this paper as [241]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Continuous control with deep reinforcement learning
Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., & Wierstra, D. (2015) · arXiv · doi:10.48550/arxiv.1509.02971 · Cited by 10,782
High performance cache replacement using re-reference interval prediction (RRIP)
Jaleel, A., Theobald, K. B., Steely, S. C., & Emer, J. (2010) · Proceedings of the 37th annual international symposium on Computer architecture · doi:10.1145/1815961.1815971 · Cited by 641
Learning-based optimization of cache content in a small cell base station
Blasco, P., & Gunduz, D. (2014) · 2014 IEEE International Conference on Communications (ICC) · doi:10.1109/icc.2014.6883600 · Cited by 298
Reinforcement Learning in Large Discrete Action Spaces.Preprint
Dulac-Arnold, G., Evans, R., Sunehag, P., & Coppin, B. (2015) · arXiv (Cornell University) · Cited by 292
Caching performance of content centric networks under multi-path routing (and more)
Rossi, D., & Rossini, G. (2011) · HAL (Le Centre pour la Communication Scientifique Directe) · Cited by 291
Deep Reinforcement Learning in Large Discrete Action Spaces
Dulac-Arnold, G., Evans, R., van Hasselt, H., Sunehag, P., Lillicrap, T., Hunt, J., Mann, T., Weber, T., Degris, T., & Coppin, B. (2015) · arXiv · doi:10.48550/arxiv.1512.07679 · Cited by 290
High performance cache replacement using re-reference interval prediction (RRIP)
Jaleel, A., Theobald, K. B., Steely, S. C., & Emer, J. (2010) · ACM SIGARCH Computer Architecture News · doi:10.1145/1816038.1815971 · Cited by 264
Popularity-driven content caching
Li, S., Xu, J., van der Schaar, M., & Li, W. (2016) · IEEE INFOCOM 2016 - The 35th Annual IEEE International Conference on Computer Communications · doi:10.1109/infocom.2016.7524381 · Cited by 158
Placing dynamic content in caches with small population
Leconte, M., Paschos, G., Gkatzikis, L., Draief, M., Vassilaras, S., & Chouvardas, S. (2016) · IEEE INFOCOM 2016 - The 35th Annual IEEE International Conference on Computer Communications · doi:10.1109/infocom.2016.7524380 · Cited by 151
Learning Based Content Caching and Sharing for Wireless Networks
Song, J., Sheng, M., Quek, T. Q. S., Xu, C., & Wang, X. (2017) · IEEE Transactions on Communications · doi:10.1109/tcomm.2017.2713384 · Cited by 144
Cited by
Items 201–231 of 231
Pushing smart caching to the edge with BayCache
He, D., Jiang, J., Yang, G., et al. (2019) · EAI International Conference on Mobile and Ubiquitous Systems: Computing, Networking and Services (MobiQuitous) · doi:10.1145/3360774.3360823 · Cited by 1
Learning-Based Content Caching and User Clustering: A Deep Deterministic Policy Gradient Approach
Chan, K.-L., & Chien, F. (2020) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp40776.2020.9053382 · Cited by 1
Artificial Intelligence for Optimizing Edge
Wang, X., Han, Y., Leung, V. C. M., Niyato, D. T., Yan, X., & Chen, X. (2020) · Springer eBooks · doi:10.1007/978-981-15-6186-3_8 · Cited by 1
Dynamic Caching for Files With Rapidly-Varying Features and Content
Amir, M., Bedeer, E. E., Khattab, T. M., et al. (2022) · IEEE Transactions on Communications · doi:10.1109/tcomm.2022.3214885 · Cited by 1
Multi-agent Deep Reinforcement Learning-Based Content Caching in Cache-Enabled Networks
Liu, S., Chen, J., Gao, D., Yang, M., & Ma, J. (2023) · Communications in computer and information science · doi:10.1007/978-981-19-9697-9_12 · Cited by 1
Viewport-Aware Deep Reinforcement Learning Approach for 360$^o$ Video CachingPreprint
Maniotis, P., & Thomos, N. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.08473 · Cited by 1
A Survey on Reinforcement Learning-Aided Caching in Mobile Edge NetworksPreprint
Νομικός, Ν., Zoupanos, S., Charalambous, T., Krikidis, I., & Petropulu, A. P. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.05564 · Cited by 1
Liu, P., Yao, C., Li, C., Zhang, S., & Li, X. (2022) · Research Square · doi:10.21203/rs.3.rs-1497139/v1 · Cited by 1
Collaborative Mobile Edge Caching Strategy Based on Deep Reinforcement Learning
Ren, J., Hou, T., & Zheng, S. (2020) · Lecture notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering · doi:10.1007/978-3-030-63941-9_1
Robust, Deep, and Reinforcement Learning for Management of Communication and Power NetworksPreprint
Sadeghi, A. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2202.05395
A Future-Oriented Cache Management for Mobile Games
Hu, Y. H., Wang, M., Chen, Y., & Fan, C. (2021) · IEEE Conference on Games (CoG) · doi:10.1109/cog52621.2021.9619023
Optimizing Cache Hit Ratio in Edge Computing Using Machine Learning Techniques
Khan, M., Wei, P. W., & Naz, F. (2022) · International Journal for Research in Applied Science and Engineering Technology · doi:10.22214/ijraset.2022.41256
Next Generation Wireless Communication: Facilitated by Machine Learning
Singh, P. K. (2022) · Lecture notes in electrical engineering · doi:10.1007/978-981-16-8892-8_59
Machine Learning Enabled Edge Computing: A Survey and Research Challenges
Xu, S., Yu, Z., Fu, K., Jia, Q., & Xie, R. (2022) · Smart innovation, systems and technologies · doi:10.1007/978-981-16-8558-3_14 · Cited by 3
Du, M., Sun, S., Ren, L., Tian, D., & Wei, A. Y. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-06794-5_45
Towards Optimising Fog Caching Using Deep Reinforcement Learning
Tajioue, M. A., Moutai, F. Z., Azzouzi, S., & Charaf, M. E. H. (2022) · IEEE International Conference on Electronics, Control, Optimization and Computer Science (ICECOCS) · doi:10.1109/icecocs55148.2022.9983128
RECC: A Relationship-Enhanced Content Caching Algorithm Using Deep Reinforcement Learning
Ren, J., Zhang, H., Zhou, X., & Zhu, M. (2022) · Asian Conference on Artificial Intelligence Technology (ACAIT) · doi:10.1109/acait56212.2022.10137967
DRL at the Application and Service Layer
Hoang, D. T., Huynh, N. V., Nguyen, D. N., Hossain, E., & Niyato, D. T. (2023) · Wiley eBooks · doi:10.1002/9781119873747.ch8
Deep Reinforcement Learning-based Edge Caching for Industrial Control Applications
Zhang, L., Xu, H., Wang, G., et al. (2023) · Chinese Control and Decision Conference (CCDC) · doi:10.1109/ccdc58219.2023.10327657
Intelligent Online Caching Strategy in 6G-Enhanced Automated Vehicular Communications
Saikia, P., Wang, G.-Y., Singh, K. K., Singh, S. K., Li, C., & Biswas, S. (2023) · IEEE International Conference on Advanced Networks and Telecommunications Systems · doi:10.1109/ants59832.2023.10468769 · Cited by 1
Adopting Deep Reinforcement Learning in Edge Caching Strategy
Feng, H., Ye, X., & Manoharan, S. (2024) · International Conference on Electrical, Computer, Communications and Mechatronics Engineering (ICECCME) · doi:10.1109/iceccme62383.2024.10796225
HCTRL: Hierarchical Clustering Tree Reinforcement Learning Algorithm for Efficient Content Caching
Shi, R., Ming, Z., Fan, Q., et al. (2025) · Lecture notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering · doi:10.1007/978-3-031-93193-2_7
A Taylor Flamingo Shark Optimization–Based Traffic Aware Content Caching Vehicular Social Networks
Vedha Vinodha, D., & Subramanium, M. (2025) · International Journal of Communication Systems · doi:10.1002/dac.70188
A Preliminary Study on the Multi-Armed Bandit-based Caching Strategy
Kayashima, S., & Nakamura, R. (2025) · IEEE International Conference on Consumer Electronics - Taiwan (ICCE-Taiwan) · doi:10.1109/icce-taiwan66881.2025.11208159
Hybrid Q-Learning Method for Content Caching in 5G Edge Networks
Shiran, S., Varposhti, M., & Nemati, S. (2025) · doi:10.1109/iot69654.2025.11297619
Hmar, L., & Chhangte, L. (2026) · Research Square · doi:10.21203/rs.3.rs-8013149/v1
Machine Learning-Based Caching and Tiering in Modern Data Storage Systems: A Survey
Savva, G., Kakoulli, E., & Herodotou, H. (2026) · ACM Computing Surveys · doi:10.1145/3803857 · Cited by 1
Online Caching Without File Splitting
Garrido, D., Moradmand, H., & Peleato, B. (2026) · IEEE Transactions on Cognitive Communications and Networking · doi:10.1109/tccn.2026.3704730
Agentic QoE Optimization for Caching-Aware Cooperative Clustering in Cell-Free Railway Systems
Zhang, Y., Xu, Z., Chen, S., et al. (2025) · IEEE Globecom Workshops (GC Wkshps) · doi:10.1109/gcwkshps68340.2025.11590873
Khuntia, P., & Hazra, R. (2026) · Telecommunication Systems · doi:10.1007/s11235-026-01528-7
Metadata
- Authors
- Zhong, C., Gursoy, M. C., & Velipasalar, S.
- Affiliations
- Syracuse University (US)
- Venue
- 2018 52nd Annual Conference on Information Sciences and Systems (CISS)
- Publisher
- IEEE
- Published
- 2018-03
- Pages
- 1-6
- OpenAlex
- W2963805280
- Semantic Scholar
- 62e318b1ca1dc63924a70e5372bf39d39ad78727
- Cited by
- 231
- References
- 15
- Citations by year
- 2026: 6 · 2025: 11 · 2024: 27 · 2023: 29 · 2022: 37 · 2021: 44 · 2020: 39 · 2019: 29 · 2018: 9
- FWCI
- 25.00
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- Conference on Information Sciences and Systems (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Networks and Communications › Caching and Content Delivery
- Physical Sciences › Computer Science › Computer Networks and Communications › IoT and Edge/Fog Computing
- Physical Sciences › Computer Science › Computer Networks and Communications › Age of Information Optimization
- Keywords
- content caching, edge computing, deep reinforcement learning, cache hit rate, base station caching, action space reduction, deep Q-network, LRU, LFU, FIFO, wireless networks, data traffic reduction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.