Caching in Base Station with Recommendation via Q-Learning
The DOI matches its Crossref record; 17 of the 19 references it cites were found in a registry or an index.
Abstract
Proactive caching in base station (BS) is a promising way to leverage the human-behavior related information to boost the system throughput and improve user experience with low cost. Yet existing caching policies are "blind" to users, i.e., the users are unaware about whether the files to be requested are locally cached at the BS around them, whose performance will degrade in mobile systems due to the randomness of user behavior. For example, when a user stays in a cell where the BS caches the file to be requested later, the user may not initiate the request. When the user sends the request, it may have entered another cell where the BS does not cache the requested file. In this paper, we introduce a simple idea of informing the users about what the BS has cached to make local caching efficient, which can be regarded as a form of recommendation. Since the probabilities that the users request the files are unknown, we resort to Q-learning to perceive the request probability and the statistics of random arrival and departure of mobile users. Then, a cache replacement policy is optimized. Simulation results demonstrate that the proposed policy with recommendation can provide higher long-term system reward than existing policies without recommendation.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Compared to traditional RL irrelevant to DL, such as Q-learning [240] and Multi-Armed Bandit (MAB) learning [230], the advantage of DRL lies in that DNNs can learn key features from the raw observation data.”
From Convergence of Edge Computing and Deep Learning: A Comprehensive Survey (2020) · cites this paper as [240]
Citations
Cites
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Web caching and Zipf-like distributions: evidence and implications
Breslau, L., Cao, P., Fan, L., Phillips, G., & Shenker, S. (1999) · IEEE INFOCOM '99. Conference on Computer Communications. Proceedings. Eighteenth Annual Joint Conference of the IEEE Computer and Communications Societies. The Future is Now (Cat. No.99CH36320) · doi:10.1109/infcom.1999.749260 · Cited by 3,467
Explaining collaborative filtering recommendations
Herlocker, J. L., Konstan, J. A., & Riedl, J. (2000) · Proceedings of the 2000 ACM conference on Computer supported cooperative work · doi:10.1145/358916.358995 · Cited by 1,731 · cited 1×
“By explaining the reason behind a recommendation, the results in [13] demonstrate that nearly five times more users are convinced by the recommendation to watch a movie, compared to the number of users who prefer the film originally.”
BackgroundThe influence of online product recommendations on consumers' online choices
Senecal, S., & Nantel, J. (2004) · Journal of Retailing · doi:10.1016/j.jretai.2004.04.001 · Cited by 1,585 · cited 1×
“The results in [12] show that recommendation can double the purchase rate of online products.”
BackgroundLiving on the edge: The role of proactive caching in 5G wireless networks
Bastug, E., Bennis, M., & Debbah, M. (2014) · IEEE Communications Magazine · doi:10.1109/mcom.2014.6871674 · Cited by 1,146
Being accurate is not enough: how accuracy metrics have hurt recommender systems
McNee, S. M., Riedl, J., & Konstan, J. A. (2006) · CHI '06 Extended Abstracts on Human Factors in Computing Systems · doi:10.1145/1125451.1125659 · Cited by 1,066
Collaborative Filtering Recommender Systems
Ekstrand, M. D. (2011) · Foundations and Trends® in Human–Computer Interaction · doi:10.1561/1100000009 · Cited by 1,037
A Course in Probability Theory
Feldman, R. E. (2002) · Journal of the American Statistical Association · doi:10.1198/jasa.2002.s482 · Cited by 573
Caching at the wireless edge: design aspects, challenges, and future directions
Liu, D., Chen, B., Yang, C., & Molisch, A. F. (2016) · IEEE Communications Magazine · doi:10.1109/mcom.2016.7565183 · Cited by 457
Self-citationAnalysis on Cache-Enabled Wireless Heterogeneous Networks
Yang, C., Yao, Y., Chen, Z., & Xia, B. (2016) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2015.2468220 · Cited by 322
Self-citation
Research works printed but not linked: 2
- shani, 2005, An MDP-based recommender system, J Machine Learning Research, 6, 1265
- 2013, Global mobile data traffic forecast update, 2012-2017, Cisco white paper
Cited by
Convergence of Edge Computing and Deep Learning: A Comprehensive Survey
Wang, X., Han, Y., Leung, V. C. M., Niyato, D., Yan, X., & Chen, X. (2020) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2020.2970550 · Cited by 1,498
“Compared to traditional RL irrelevant to DL, such as Q-learning [240] and Multi-Armed Bandit (MAB) learning [230], the advantage of DRL lies in that DNNs can learn key features from the raw observation data.”
BackgroundWei, Y., Yu, F. R., Song, M., & Han, Z. (2019) · IEEE Internet of Things Journal · doi:10.1109/jiot.2018.2878435 · Cited by 321
Learn to Cache: Machine Learning for Network Edge Caching in the Big Data Era
Chang, Z., Lei, L., Zhou, Z., Mao, S., & Ristaniemi, T. (2018) · IEEE Wireless Communications · doi:10.1109/mwc.2018.1700317 · Cited by 212
AI-Based Mobile Edge Computing for IoT: Applications, Challenges, and Future Scope
Singh, A. P., Satapathy, S. C., Roy, A., & Gutub, A. (2022) · Arabian Journal for Science and Engineering · doi:10.1007/s13369-021-06348-2 · Cited by 169
Intelligent Content Caching Strategy in Autonomous Driving Toward 6G
Zhao, L., Li, H., Lin, N., Lin, M., Fan, C., & Shi, J. (2021) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2021.3114199 · Cited by 87
Tang, J., Tang, H., Zhang, X. Y., Cumanan, K., Chen, G., Wong, K., & Chambers, J. A. (2019) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2019.2954127 · Cited by 70
Collaborative Multi-Agent Multi-Armed Bandit Learning for Small-Cell Caching
Xu, X., Tao, M., & Shen, C. (2020) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2020.2966599 · Cited by 64
Low-latency edge cooperation caching based on base station cooperation in SDN based MEC
Li, C., Cai, Q., & Luo, Y. (2021) · Expert Systems with Applications · doi:10.1016/j.eswa.2021.116252 · Cited by 53
A Learning-Based Approach to Joint Content Caching and Recommendation at Base Stations
Liu, D., & Yang, C. (2018) · Globecom · doi:10.1109/glocom.2018.8647827 · Cited by 52
Self-citationUsing Grouped Linear Prediction and Accelerated Reinforcement Learning for Online Content Caching
Zhang, N., Zheng, K., & Tao, M. (2018) · IEEE International Conference on Communications workshops · doi:10.1109/iccw.2018.8403712 · Cited by 54
Deep Q-Learning-Based Content Caching With Update Strategy for Fog Radio Access Networks
Jiang, F., Yuan, Z., Sun, C., & Wang, J. (2019) · IEEE Access · doi:10.1109/access.2019.2927836 · Cited by 52
Liu, D., & Yang, C. (2019) · IEEE Access · doi:10.1109/access.2019.2925019 · Cited by 50
Self-citationA Novel Deep Q-Learning-Based Air-Assisted Vehicular Caching Scheme for Safe Autonomous Driving
Shi, J., Zhao, L., Wang, X., Zhao, W., Hawbani, A., & Huang, M. (2020) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2020.3018720 · Cited by 41
Naeem, M. A., Zikria, Y. B., Ali, R., Tariq, U., Meng, Y., & Bashir, A. K. (2022) · Digital Communications and Networks · doi:10.1016/j.dcan.2022.08.004 · Cited by 38
Fu, Y., Zhang, Y., Wong, K.-Y., & Quek, T. Q. S. (2022) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2022.3142809 · Cited by 31
Optimizing Caching and Recommendation Towards User Satisfaction
Qi, K., Chen, B., Yang, C., & Han, S. (2018) · International Conference on Wireless Communications and Signal Processing (WCSP) · doi:10.1109/wcsp.2018.8555592 · Cited by 29
Self-citationLiu, Y., Jia, J., Cai, J., & Huang, T. (2022) · IEEE Transactions on Network and Service Management · doi:10.1109/tnsm.2022.3207994 · Cited by 30
An Artificial Intelligence Perspective on Mobile Edge Computing
Chen, Z., He, Q., Liu, L., Lan, D., Chung, H.-M., & Mao, Z. (2019) · IEEE International Conference on Smart Internet of Things (SmartIoT) · doi:10.1109/smartiot.2019.00024 · Cited by 26
Caching in Dynamic Environments: A Near-Optimal Online Learning Approach
Zhou, S., Wang, Z., Hu, C., Mao, Y., Yan, H., Zhang, S., Wu, C., & Zhu, W. (2021) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2021.3132156 · Cited by 28
Efficient caching method in fog computing for internet of everything
Riya, Gupta, N., & Dhurandher, S. K. (2020) · Peer-to-Peer Networking and Applications · doi:10.1007/s12083-020-00952-z · Cited by 24
Energy and Delay Optimization for Cache-Enabled Dense Small Cell Networks
Wu, H., Lu, H., Wu, F., & Chen, C. W. (2020) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2020.2989033 · Cited by 27
Distributed Edge Caching with Content Recommendation in Fog-RANs Via Deep Reinforcement Learning
Yan, J., Jiang, Y., Zheng, F., Yu, F. R., Gao, X., & You, X. (2020) · IEEE International Conference on Communications workshops · doi:10.1109/iccworkshops49005.2020.9145039 · Cited by 24
Temporal-Spatial Recommendation for Caching at Base Stations via Deep Reinforcement Learning
Guo, K., & Yang, C. (2019) · IEEE Access · doi:10.1109/access.2019.2914500 · Cited by 22
Self-citationDecentralized Multi-Agent Multi-Armed Bandit Learning With Calibration for Multi-Cell Caching
Xu, X., & Tao, M. (2020) · IEEE Transactions on Communications · doi:10.1109/tcomm.2020.3045050 · Cited by 21
Yan, J., Zhang, M., Jiang, Y., Zheng, F., Chang, Q., Abualnaja, K. M., Mumtaz, S., & You, X. (2023) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2023.3328554 · Cited by 17
Joint Pushing and Recommendation for Susceptible Users with Time-Varying Connectivity
Lin, Z., & Chen, W. (2018) · Globecom · doi:10.1109/glocom.2018.8647838 · Cited by 17
Lin, Z., & Chen, W. (2019) · IEEE Transactions on Communications · doi:10.1109/tcomm.2019.2929129 · Cited by 18
Collaborative Multi-Agent Reinforcement Learning of Caching Optimization in Small-Cell Networks
Xu, X., & Tao, M. (2018) · Globecom · doi:10.1109/glocom.2018.8647341 · Cited by 12
Network Friendly Recommendations: Optimizing for Long Viewing Sessions
Γιαννακάς, Θ., Sermpezis, P., & Spyropoulos, T. (2021) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2021.3109727 · Cited by 14
Increasing network throughput based on dynamic caching policy at wireless access points
Ren, J., Hou, T., Wang, H., Ren, H., & Zhang, X. (2019) · Wireless Networks · doi:10.1007/s11276-019-02125-0 · Cited by 12
Caching With Personalized and Incumbent-Aware Recommendation: Modeling and Optimization
Zhao, Y., Yu, Z., & Yuan, D. (2024) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2024.3365465 · Cited by 13
Vinh, Q., & Koo, I. (2019) · IEEE Systems Journal · doi:10.1109/jsyst.2019.2958993 · Cited by 11
Network Coding-Based Socially-Aware Caching Strategy in D2D
Cai, J., Wu, X., Liu, Y., Luo, J., & Liao, L. (2020) · IEEE Access · doi:10.1109/access.2020.2966289 · Cited by 9
Monte-Carlo Tree Search Aided Contextual Online Learning Approach for Wireless Caching
Du, Y., Gao, P., Wang, X., Dong, B., Chen, Z., & Li, S. (2018) · IEEE Globecom Workshops (GC Wkshps) · doi:10.1109/glocomw.2018.8644489 · Cited by 9
Liu, X., Derakhshani, M., & Lambotharan, S. (2021) · IEEE Transactions on Communications · doi:10.1109/tcomm.2021.3059305 · Cited by 8
Energy Optimization of a Base Station using Q-learning Algorithm
Aggarwal, A., & Selvamuthu, D. (2023) · doi:10.1109/contel58387.2023.10198963 · Cited by 8
Popularity based content caching of YouTube data in cellular networks
Raja, K., Abilash, B., Anbalagan, S., Dev, K., & Ganapathisubramaniyan, A. (2022) · Multimedia Tools and Applications · doi:10.1007/s11042-022-13528-1 · Cited by 6
IBP Based Caching Strategy in D2D
Shan, C., Wu, X., Liu, Y., Cai, J., & Luo, J. (2019) · Applied Sciences · doi:10.3390/app9122416 · Cited by 5
Yang, M.-H., Lee, M., & Hong, Y.-W. P. (2024) · IEEE Transactions on Communications · doi:10.1109/tcomm.2024.3447891 · Cited by 4
Zhang, J., & Wang, J. (2023) · Neurocomputing · doi:10.1016/j.neucom.2023.126258 · Cited by 5
A Privacy-Preserving Caching Scheme for Device-to-Device Communications
Zhong, Y., Li, Z., & Liao, L. (2021) · Security and Communication Networks · doi:10.1155/2021/6696149 · Cited by 5
Learning-Based Content Caching with Update Strategy for Fog Radio Access Networks
Jiang, F., Yuan, Z., Sun, C., Ren, Y., & Wang, J. (2019) · IEEE/CIC International Conference on Communications in China (ICCC) · doi:10.1109/iccchina.2019.8855843 · Cited by 3
Data Caching at Fog Nodes Under IoT Networks: Review of Machine Learning ApproachesPreprint
Tapwal, R., Gupta, N., & Xin, Q. (2020) · doi:10.36227/techrxiv.12091410.v1 · Cited by 3
Shan, Y., Wang, H., & Zhang, C. (2022) · Security and Communication Networks · doi:10.1155/2022/2965081 · Cited by 3
Dynamic Edge Caching via Online Meta-RL
Mao, Y., Zhou, S., Liu, H., Wang, Z., & Zhu, W. (2023) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn54540.2023.10191608 · Cited by 2
Data Caching at Fog Nodes Under IoT Networks: Review of Machine Learning ApproachesPreprint
Tapwal, R., Gupta, N., & Xin, Q. (2020) · doi:10.36227/techrxiv.12091410 · Cited by 3
Content Recommendation Algorithm Based on Double Lists in Heterogeneous Network
Chen, J., Li, X., Ji, H., & Zhang, H. (2020) · Lecture notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering · doi:10.1007/978-3-030-41117-6_12 · Cited by 1
Content-Aware AP Selection with LSTM-enabled Proactive Caching in Cell-Free Massive MIMO Networks
Ajmal, M., Park, S., Saad, M. M., Tariq, M. A., & Kim, D. (2025) · IEEE Transactions on Network Science and Engineering · doi:10.1109/tnse.2025.3578687 · Cited by 1
Artificial Intelligence for Optimizing Edge
Wang, X., Han, Y., Leung, V. C. M., Niyato, D. T., Yan, X., & Chen, X. (2020) · Springer eBooks · doi:10.1007/978-981-15-6186-3_8 · Cited by 1
Yousefi, H. (2018) · Scholarship@Western (Western University)
Network Friendly Recommendations: Optimizing for Long Viewing SessionsPreprint
Γιαννακάς, Θ., Sermpezis, P., & Spyropoulos, T. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.00772
Towards Optimising Fog Caching Using Deep Reinforcement Learning
Tajioue, M. A., Moutai, F. Z., Azzouzi, S., & Charaf, M. E. H. (2022) · IEEE International Conference on Electronics, Control, Optimization and Computer Science (ICECOCS) · doi:10.1109/icecocs55148.2022.9983128
Xiao, J., Lin, J., Wen, J., Yang, C., Tang, Y., Chen, X., & Liu, T. (2026) · Lecture notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering · doi:10.1007/978-3-032-23456-8_11
Self-citation
Metadata
- Authors
- Guo, K., Yang, C., & Liu, T.
- Affiliations
- Beihang University (CN)
- Venue
- 2017 IEEE Wireless Communications and Networking Conference (WCNC)
- Publisher
- IEEE
- Published
- 2017-03
- Pages
- 1-6
- ISSN
- 1525-3511 (ISSN-L)
- OpenAlex
- W2612386806
- Cited by
- 54
- References
- 19
- Citations by year
- 2026: 1 · 2025: 1 · 2024: 2 · 2023: 4 · 2022: 7 · 2021: 7 · 2020: 13 · 2019: 10 · 2018: 9
- FWCI
- 6.97
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE Wireless Communications and Networking Conference (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Networks and Communications › Caching and Content Delivery
- Physical Sciences › Computer Science › Computer Networks and Communications › IoT and Edge/Fog Computing
- Physical Sciences › Computer Science › Computer Networks and Communications › Age of Information Optimization
- Keywords
- proactive caching, base station caching, user recommendation, Q-learning, cache replacement policy, user mobility
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.