The DOI matches its Crossref record; 50 of the 50 references it cites were found in a registry or an index.
Mobility-Aware Edge Caching and Computing in Vehicle Networks: A Deep Reinforcement Learning
Abstract
This paper studies the joint communication, caching and computing design problem for achieving the operational excellence and the cost efficiency of the vehicular networks. Moreover, the resource allocation policy is designed by considering the vehicle's mobility and the hard service deadline constraint. These critical challenges have often been either neglected or addressed inadequately in the existing work on the vehicular networks because of their high complexity. We develop a deep reinforcement learning with the multi-timescale framework to tackle these grand challenges in this paper. Furthermore, we propose the mobility-aware reward estimation for the large timescale model to mitigate the complexity due to the large action space. Numerical results are presented to illustrate the theoretical findings developed in the paper and to quantify the performance gains attained.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Besides, Vehicle-to-Vehicle (V2V) communication technology can be taken advantaged to further connect vehicles, either as an edge node or an end device managed by DRL-based control policies [112].”
From Convergence of Edge Computing and Deep Learning: A Comprehensive Survey (2020) · cites this paper as [112]
Citations
Cites
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357 · cited 5×
“Interested readers can find detailed information in [34].”
BackgroundMethodInfluentialTensorFlow: A system for large-scale machine learning
Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J., Devin, M., Ghemawat, S., Irving, G., Isard, M., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.08695 · Cited by 11,822
Introduction to Probability Models.
Jolliffe, F., & Ross, S. M. (1995) · Journal of the Royal Statistical Society. Series A (Statistics in Society) · doi:10.2307/2983433 · Cited by 4,525
Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning
Sutton, R. S., Precup, D., & Singh, S. (1999) · Artificial Intelligence · doi:10.1016/s0004-3702(99)00052-1 · Cited by 3,119 · cited 2×
“However, the proposed algorithms in [47], [48] are only efficient for the small size networks.”
BackgroundVehicular Fog Computing: A Viewpoint of Vehicles as the Infrastructures
Hou, X., Li, Y., Chen, M., Wu, D., Jin, D., & Chen, S. (2016) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2016.2532863 · Cited by 930 · cited 1×
“Recently, edge caching placements and computing offloading at the vehicles and the road side units (RSUs) have been proposed to efficiently improve the QoS for applications with intensive computations and hard deadlines [6], [7].”
BackgroundFadlullah, Z. M., Tang, F., Mao, B., Kato, N., Akashi, O., Inoue, T., & Mizutani, K. (2017) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2017.2707140 · Cited by 810
Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivation
Kulkarni, T. D., Narasimhan, K., Saeedi, A., & Tenenbaum, J. B. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1604.06057 · Cited by 770 · cited 3×
“Based on [46]–[48], we propose a multi-timescale system framework, where the large timescale Q-learning model is for every epoch (corresponding to Td time slots) and the small timescale Q-learning model is for every time slot.”
BackgroundFog-computing-based radio access networks: issues and challenges
Peng, M., Yan, S., Zhang, K., & Wang, C. (2016) · IEEE Network · doi:10.1109/mnet.2016.7513863 · Cited by 660
Introduction to Probability Models
(2014) · Introduction to Probability Models · doi:10.1016/b978-0-12-407948-9.00013-x · Cited by 653
Fundamental Limits of Caching in Wireless D2D Networks
Ji, M., Caire, G., & Molisch, A. F. (2016) · IEEE Transactions on Information Theory · doi:10.1109/tit.2015.2504556 · Cited by 596 · cited 1×
“It can cache popular contents at various nearby devices [11].”
BackgroundHe, Y., Zhao, N., & Yin, H. (2018) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2017.2760281 · Cited by 569 · cited 2×
“Machine learning is an emerging tool to tackle problems encountered in caching, computing and communications in 5G wireless communications [21], [23], [26], [30]–[40].”
BackgroundAnalyzing the Video Popularity Characteristics of Large-Scale User Generated Content Systems
Cha, M., Kwak, H., Rodriguez, P., Ahn, Y.-Y., & Moon, S. (2009) · IEEE/ACM Transactions on Networking · doi:10.1109/tnet.2008.2011358 · Cited by 485 · cited 2×
“Furthermore, recent studies reveal that different contents require different levels of priority [4], [5].”
BackgroundAn energy efficient and spectrum efficient wireless heterogeneous network framework for 5G systems
Hu, R. Q., & Qian, Y. (2014) · IEEE Communications Magazine · doi:10.1109/mcom.2014.6815898 · Cited by 445 · cited 6×
“Recently, edge caching placements and computing offloading at the vehicles and the road side units (RSUs) have been proposed to efficiently improve the QoS for applications with intensive computations and hard deadlines [6], [7].”
BackgroundSelf-citationKato, N., Fadlullah, Z. M., Mao, B., Tang, F., Akashi, O., Inoue, T., & Mizutani, K. (2017) · IEEE Wireless Communications · doi:10.1109/mwc.2016.1600317wc · Cited by 400
Rodrigues, T. G., Suto, K., Nishiyama, H., & Kato, N. (2017) · IEEE Transactions on Computers · doi:10.1109/tc.2016.2620469 · Cited by 374 · cited 2×
“2867191 and management from the centralized cloud to the edge of the network [8]–[10].”
BackgroundHe, Y., Yu, F. R., Zhao, N., Leung, V. C. M., & Yin, H. (2017) · IEEE Communications Magazine · doi:10.1109/mcom.2017.1700246 · Cited by 358 · cited 2×
“Machine learning is an emerging tool to tackle problems encountered in caching, computing and communications in 5G wireless communications [21], [23], [26], [30]–[40].”
BackgroundContinuous Deep Q-Learning with Model-based Acceleration
Gu, S., Lillicrap, T., Sutskever, I., & Levine, S. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1603.00748 · Cited by 304 · cited 2×
“[35] can be used to approximate the reward functions.”
MethodWei, L., Hu, R., Qian, Y., & Wu, G. (2014) · IEEE Communications Magazine · doi:10.1109/mcom.2014.6829950 · Cited by 282
Self-citationAnalysis of Web caching architectures: hierarchical and distributed caching
Rodriguez, P., Spanner, C., & Biersack, E. W. (2001) · IEEE/ACM Transactions on Networking · doi:10.1109/90.944339 · Cited by 227 · cited 1×
“Furthermore, recent studies reveal that different contents require different levels of priority [4], [5].”
BackgroundA Learning-Based Approach to Caching in Heterogenous Small Cell Networks
Bharath, B. N., Nagananda, K. G., & Poor, H. V. (2016) · IEEE Transactions on Communications · doi:10.1109/tcomm.2016.2536728 · Cited by 202 · cited 2×
“Machine learning is an emerging tool to tackle problems encountered in caching, computing and communications in 5G wireless communications [21], [23], [26], [30]–[40].”
BackgroundMethodLiu, J., Nishiyama, H., Kato, N., & Guo, J. (2016) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2015.2452492 · Cited by 201 · cited 1×
“In general, the integration of D2D communications into cellular networks helps improve cell coverage, spectral efficiency as well as traffic congestion [12]–[15].”
BackgroundMobility-aware caching for content-centric wireless networks: modeling and methodology
Wang, R., Peng, X., Zhang, J., & Letaief, K. B. (2016) · IEEE Communications Magazine · doi:10.1109/mcom.2016.7537180 · Cited by 190 · cited 2×
“The authors in [27], [29] developed a framework of mobility-aware coded caching.”
BackgroundA Q-learning-based dynamic channel assignment technique for mobile communication systems
Nie, J., & Haykin, S. (1999) · IEEE Transactions on Vehicular Technology · doi:10.1109/25.790549 · Cited by 181 · cited 3×
“Furthermore, Qt(x, a) definitely converges toQ∗(x, a), when a properα is selected [36].”
BackgroundMethodPricing and Resource Allocation via Game Theory for a Small-Cell Video Caching System
Li, J., Chen, H., Chen, Y., Lin, Z., Vucetic, B. S., & Hanzo, L. (2016) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2016.2577278 · Cited by 167 · cited 2×
“In the recent information-centric networking paradigm, cooperative caching constitutes one of the most widely studied wireless caching paradigms, where local user terminals and relay nodes can cooperatively store the multimedia contents [16]–[20], [23]– [26].”
BackgroundResultZhou, Y., Yu, F. R., Chen, J., & Kuo, Y. (2017) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2017.2737028 · Cited by 165 · cited 2×
“In the recent information-centric networking paradigm, cooperative caching constitutes one of the most widely studied wireless caching paradigms, where local user terminals and relay nodes can cooperatively store the multimedia contents [16]–[20], [23]– [26].”
BackgroundMobility-Aware Caching in D2D Networks
Wang, R., Zhang, J., Song, S. H., & Letaief, K. B. (2017) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2017.2705038 · Cited by 158
Wang, C., He, Y., Yu, F. R., Chen, Q., & Tang, L. (2017) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2017.2758763 · Cited by 156 · cited 1×
“To tackle these grant challenges, advanced technologies are required for enhancing the capacity and QoS [1]–[3].”
BackgroundWei, Y., Yu, F. R., & Song, M. N. (2010) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2010.2041803 · Cited by 138
Distributed Caching for Data Dissemination in the Downlink of Heterogeneous Networks
Li, J., Chen, Y., Lin, Z., Chen, W., Vucetic, B. S., & Hanzo, L. (2015) · IEEE Transactions on Communications · doi:10.1109/tcomm.2015.2455500 · Cited by 137 · cited 1×
“Relevant results reported in some recent works such as those in [16], [18], [19] would be useful for these further studies.”
ResultDisaster Management Using D2D Communication With Power Transfer and Clustering Techniques
Ali, K., Nguyen, H. X., Vien, Q., Shah, P., & Chu, Z. (2018) · IEEE Access · doi:10.1109/access.2018.2793532 · Cited by 121 · cited 1×
“In general, the integration of D2D communications into cellular networks helps improve cell coverage, spectral efficiency as well as traffic congestion [12]–[15].”
BackgroundHarnessing cloud and edge synergies: toward an information theory of fog radio access networks
Tandon, R., & Simeone, O. (2016) · IEEE Communications Magazine · doi:10.1109/mcom.2016.7537176 · Cited by 109 · cited 1×
“2867191 and management from the centralized cloud to the edge of the network [8]–[10].”
BackgroundAdaptive Scheme for Caching YouTube Content in a Cellular Network: Machine Learning Approach
Tanzil, S. M. S., Hoiles, W., & Krishnamurthy, V. (2017) · IEEE Access · doi:10.1109/access.2017.2678990 · Cited by 106 · cited 1×
“[32] introduced a novel scheme for estimating the unknown popularity of caching contents, namely an extreme-learning machine framework.”
BackgroundLiu, K., Ng, J. K., Wang, J., Lee, V. C. S., Wu, W., & Son, S. H. (2015) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2015.2495269 · Cited by 99 · cited 1×
“Recently, edge caching placements and computing offloading at the vehicles and the road side units (RSUs) have been proposed to efficiently improve the QoS for applications with intensive computations and hard deadlines [6], [7].”
BackgroundCode, Cache and Deliver on the Move: A Novel Caching Paradigm in Hyper-Dense Small-Cell Networks
Poularakis, K., & Tassiulas, L. (2016) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2016.2575837 · Cited by 99 · cited 1×
“The authors in [28] worked on minimizing the workload at macro BSs by making the strategy of content caching at the small-cell base stations when the user’s mobility is considered.”
BackgroundNetwork-Assisted Outband D2D-Clustering in 5G Cellular Networks: Theory and Practice
Asadi, A., & Mancuso, V. (2016) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2016.2621041 · Cited by 99
Probabilistic Small-Cell Caching: Performance Analysis and Optimization
Chen, Y., Ding, M., Li, J., Lin, Z., Mao, G., & Hanzo, L. (2016) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2016.2606765 · Cited by 98 · cited 1×
“Relevant results reported in some recent works such as those in [16], [18], [19] would be useful for these further studies.”
ResultEdge Caching at Base Stations With Device-to-Device Offloading
Wang, W., Lan, R., Gu, J., Huang, A., Shan, H., & Zhang, Z. (2017) · IEEE Access · doi:10.1109/access.2017.2679198 · Cited by 91 · cited 1×
“[31] proposed a joint BS caching and D2D offloading using Q-learning.”
MethodDistributed Deep Q-LearningPreprint
Ong, H. Y., Chavez, K., & Hong, A. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1508.04186 · Cited by 83 · cited 4×
“3) We develop the algorithm based on deep Q-learning with multi-timescale framework [33]–[36] to configure the parameters of caching placement and computing resource allocation as well as to determine the sets of possible connecting RSUs and neighboring vehicles.”
BackgroundMethodInfluentialMultitime scale Markov decision processes
Chang, H. S., Fard, P. J., Marcus, S. I., & Shayman, M. A. (2003) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2003.812782 · Cited by 83 · cited 1×
“Based on [46]–[48], we propose a multi-timescale system framework, where the large timescale Q-learning model is for every epoch (corresponding to Td time slots) and the small timescale Q-learning model is for every time slot.”
BackgroundCache-Enabled Adaptive Video Streaming Over Vehicular Networks: A Dynamic Approach
Guo, Y., Yang, Q., Yu, F. R., & Leung, V. C. M. (2018) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2018.2817210 · Cited by 80
Joint Resource Allocation for Software-Defined Networking, Caching, and Computing
Chen, Q., Yu, F. R., Huang, T., Xie, R., Liu, J., & Liu, Y. (2018) · IEEE/ACM Transactions on Networking · doi:10.1109/tnet.2017.2782216 · Cited by 78
Estimating Instantaneous Cache Hit Ratio Using Markov Chain Analysis
Gomaa, H., Messier, G. G., Williamson, C. L., & Davies, R. (2012) · IEEE/ACM Transactions on Networking · doi:10.1109/tnet.2012.2227338 · Cited by 59 · cited 2×
“So the transition probability matrix can be written as [43]”
BackgroundMethodTan, L. T., & Le, L. B. (2015) · IEEE Access · doi:10.1109/access.2015.2509449 · Cited by 40 · cited 2×
“Machine learning is an emerging tool to tackle problems encountered in caching, computing and communications in 5G wireless communications [21], [23], [26], [30]–[40].”
BackgroundMethodSelf-citationHu, J., Yang, L., & Hanzo, L. (2017) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2017.2678167 · Cited by 29
Cooperative Data Offload in Opportunistic Networks: From Mobile Devices to Infrastructure
Lu, Z., Sun, X., & Porta, T. L. (2017) · IEEE/ACM Transactions on Networking · doi:10.1109/tnet.2017.2747621 · Cited by 23 · cited 3×
“Then, we can use WelchSatterthwaite estimation [45] to obtain the PDF of fV c i (Z,W )(v), where V c i (Z,W ) ∝ Γ(γ4, σ4) and”
MethodInfluentialJoint data compression and MAC protocol design for smartgrids with renewable energy
Tan, L. T., & Le, L. B. (2016) · Wireless Communications and Mobile Computing · doi:10.1002/wcm.2710 · Cited by 22
Self-citationECO-M: Energy-efficient Cluster-Oriented Multimedia delivery in a LTE D2D environment
Trestian, R., Vien, Q., Nguyen, H. X., & Gemikonakli, O. (2015) · International Conference on Communications · doi:10.1109/icc.2015.7248298 · Cited by 20
Compressed sensing based data processing and MAC protocol design for smartgrids
Tan, L. T., & Le, L. B. (2015) · IEEE Wireless Communications and Networking Conference · doi:10.1109/wcnc.2015.7127798 · Cited by 17 · cited 1×
“Machine learning is an emerging tool to tackle problems encountered in caching, computing and communications in 5G wireless communications [21], [23], [26], [30]–[40].”
BackgroundSelf-citationA novel and efficient mixed-signal compressed sensing for wide-band cognitive radio
Tan, L. T., & Kong, H. Y. (2010) · International Forum on Strategic Technology · doi:10.1109/ifost.2010.5668104 · Cited by 14
Self-citationTan, L. T., Ying, L., & Bliss, D. W. (2017) · Conference on Information Sciences and Systems · doi:10.1109/ciss.2017.7926066 · Cited by 14 · cited 1×
“upload its tasks to the contact RSUs/vehicles/BS by employing the full-duplex radio [40], [41].”
MethodSelf-citation
Cited by
Applications of Deep Reinforcement Learning in Communications and Networking: A Survey
Luong, N. C., Hoang, D. T., Gong, S., Niyato, D., Wang, P., Liang, Y.-C., & Kim, D. I. (2019) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2019.2916583 · Cited by 2,060
Towards 6G wireless communication networks: vision, enabling technologies, and new paradigm shifts
You, X., Wang, C.-X., Huang, J., Gao, X., Zhang, Z., Wang, M., Huang, Y., Zhang, C., Jiang, Y., Wang, J., et al. (2021) · Science China Information Sciences · doi:10.1007/s11432-020-2955-6 · Cited by 2,027
Convergence of Edge Computing and Deep Learning: A Comprehensive Survey
Wang, X., Han, Y., Leung, V. C. M., Niyato, D., Yan, X., & Chen, X. (2020) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2020.2970550 · Cited by 1,498
“Besides, Vehicle-to-Vehicle (V2V) communication technology can be taken advantaged to further connect vehicles, either as an edge node or an end device managed by DRL-based control policies [112].”
BackgroundPham, Q.-V., Fang, F., Ha, V. N., Piran, M. J., Le, M., Le, L. B., Hwang, W.-J., & Ding, Z. (2020) · IEEE Access · doi:10.1109/access.2020.3001277 · Cited by 870
“Other studies on (deep) RL based caching algorithms can be found in [246], [247].”
Quantum Machine Learning for 6G Communication Networks: State-of-the-Art and Vision for the Future
Nawaz, S. J., Sharma, S. K., Wyne, S., Patwary, M. N., & Asaduzzaman, M. (2019) · IEEE Access · doi:10.1109/access.2019.2909490 · Cited by 658
“Moreover, ML-based proactive content caching at the edge node can also be exploited to avoid backhaul congestion and further reduce the service latency [66], [67].”
Background
Metadata
- Authors
- Tan, L. T., & Hu, R. Q.
- Affiliations
- Utah State University (US)
- Venue
- IEEE Transactions on Vehicular Technology
- Published
- 2018-11
- Volume
- 67
- Issue
- 11
- Pages
- 10190-10203
- ISSN
- 0018-9545 (ISSN-L)
- 1939-9359
- OpenAlex
- W2889540440
- Cited by
- 356
- References
- 50
- Citations by year
- 2026: 9 · 2025: 15 · 2024: 30 · 2023: 40 · 2022: 65 · 2021: 61 · 2020: 95 · 2019: 36 · 2018: 3
- FWCI
- 30.19
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- IEEE Transactions on Vehicular Technology (publishedVersion)
- Funders
- National Science Foundation (NeTS-1423348, EARS-1547312)
- Topics
- Physical Sciences › Computer Science › Computer Networks and Communications › IoT and Edge/Fog Computing
- Physical Sciences › Computer Science › Computer Networks and Communications › Caching and Content Delivery
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Vehicular Ad Hoc Networks (VANETs)
- Keywords
- vehicular networks, edge caching, edge computing, vehicle mobility, resource allocation, deep reinforcement learning, large action spaces
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.