Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat
Sign Up
Cited by 2,060

The DOI matches its Crossref record; 27 of the 27 references it cites were found in a registry or an index.

Applications of Deep Reinforcement Learning in Communications and Networking: A Survey

Luong, Nguyen Cong, Hoang, Dinh Thai, Gong, Shimin, Niyato, Dusit, Wang, Ping, Liang, Ying-Chang, Kim, Dong In·2019·IEEE Communications Surveys & Tutorials
DOI 10.1109/comst.2019.2916583
Open this paper in CoChat — free account

Abstract

This paper presents a comprehensive literature review on applications of deep reinforcement learning (DRL) in communications and networking. Modern networks, e.g., Internet of Things (IoT) and unmanned aerial vehicle (UAV) networks, become more decentralized and autonomous. In such networks, network entities need to make decisions locally to maximize the network performance under uncertainty of network environment. Reinforcement learning has been efficiently used to enable the network entities to obtain the optimal policy including, e.g., decisions or actions, given their states when the state and action spaces are small. However, in complex and large-scale networks, the state and action spaces are usually large, and the reinforcement learning may not be able to find the optimal policy in reasonable time. Therefore, DRL, a combination of reinforcement learning with deep learning, has been developed to overcome the shortcomings. In this survey, we first give a tutorial of DRL from fundamental concepts to advanced models. Then, we review DRL approaches proposed to address emerging issues in communications and networking. The issues include dynamic network access, data rate control, wireless caching, data offloading, network security, and connectivity preservation which are all important to next generation networks, such as 5G and beyond. Furthermore, we present applications of DRL for traffic routing, resource sharing, and data collection. Finally, we highlight important challenges, open issues, and future research directions of applying DRL.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/comst.2019.2916583

Cites

27 of 27 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 27 references

Research works printed but not linked: 173

  1. 10.1007/BF02551385 · DOI
  2. 10.1109/WAINA.2017.67 · DOI
  3. 10.1109/TWC.2017.2768423 · DOI · Paper page
  4. 10.1145/2750858.2807513 · DOI · Paper page
  5. 10.1109/ICDCS.2018.00164 · DOI
  6. 10.1145/2487575.2488188 · DOI · Paper page
  7. 10.1145/1689239.1689247 · DOI · Paper page
  8. bracciale, 2014, CRAWDAD Dataset Roma/Taxi (v 2014-07-17)
  9. 10.1109/TII.2017.2783439 · DOI · Paper page
  10. 10.1109/TSP.2017.2673813 · DOI · Paper page
  11. 10.1109/ACSSC.2018.8645343 · DOI
  12. 10.1109/SSCI.2016.7849837 · DOI
  13. hausknecht, 2015, Deep recurrent Q-learning for partially observable MDPS, CoRR, abs 1507 6527, 1
  14. bellemare, 2017, A distributional perspective on reinforcement learning, Proc Int Conf Mach Learn, 449
  15. wang, 2018, Handover control optimization via asynchronous multi-user deep reinforcement learning, Proc IEEE ICC, 1
  16. wang, 2016, Dueling network architectures for deep reinforcement learning, Proc Int Conf Mach Learn, 1
  17. silver, 2014, Deterministic policy gradient algorithms, Proc ICML, 387
  18. fortunato, 2018, Noisy networks for exploration, Proc Int Conf Learn Represent, 1
  19. 10.1109/JIOT.2017.2712560 · DOI · Paper page
  20. 10.1016/j.jnca.2016.11.015 · DOI
  21. 10.1109/LWC.2017.2757490 · DOI · Paper page
  22. 10.1109/TVT.2018.2851783 · DOI · Paper page
  23. kingma, 2014, Semi-supervised learning with deep generative models, Proc Adv Neural Inf Process Syst, 3581
  24. 10.1109/ICCCN.2012.6289200 · DOI · Paper page
  25. 10.1109/TVT.2016.2545523 · DOI · Paper page
  26. 10.1002/9780470400531.eorms0118 · doi:10.1002/9780470400531.eorms0118
  27. 10.1109/GLOCOM.2018.8647611 · DOI
  28. 10.1109/GLOCOM.2017.8254101 · DOI
  29. 10.1109/TIFS.2017.2737968 · DOI · Paper page
  30. schaul, 2016, Prioritized experience replay, arXiv preprint arXiv 1511 05952, 1
  31. 10.1109/CVPR.2015.7298878 · DOI · Paper page
  32. lin, 2018, An efficient deep reinforcement learning model for urban traffic control, arXiv preprint arXiv 1808 02194, 1
  33. mnih, 2015, Human-level control through deep reinforcement learning, Nature, 518, 529, 10.1038/nature14236 · DOI · Paper page
  34. thrun, 1993, Issues in using function approximation for reinforcement learning, Proc Connectionist Models Summer Sch, 1
  35. 10.1109/ICRA.2017.7989385 · DOI · Paper page
  36. hasselt, 2010, Double Q-learning, Proc Adv Neural Inf Process Syst, 2613
  37. 10.1109/JIOT.2018.2872440 · DOI · Paper page
  38. 10.1109/CCAAW.2017.8001607 · DOI
  39. 10.1109/ICC.2017.7997286 · DOI · Paper page
  40. 10.1109/TSP.2018.2812733 · DOI · Paper page
  41. challita, 2018, Cellular-connected UAVs over 5G: Deep reinforcement learning for interference management, arXiv preprint arXiv 1801 01000, 1
  42. 10.1109/ACCESS.2018.2828859 · DOI
  43. 10.1109/TVT.2017.2724060 · DOI
  44. 10.1007/978-3-642-12331-3 · DOI · Paper page
  45. 10.1109/MASCOT.2001.948886 · DOI · Paper page
  46. sutton, 1999, Policy gradient methods for reinforcement learning with function approximation, Proc 12th Int Conf Neural Inf Process Syst, 1057
  47. 10.1109/ICC.2018.8422706 · DOI
  48. 10.1109/GLOCOM.2017.8254746 · DOI
  49. 10.1109/JSAC.2015.2416990 · DOI
  50. 10.1109/ACCESS.2018.2809581 · DOI
  51. 10.1155/2010/876216 · DOI
  52. balazinska, 2003, IBM Watson Research Center
  53. challita, 2017, Proactive resource management in LTE-U systems: A deep learning perspective, arXiv preprint arXiv 1702 03180, 1
  54. 10.1109/ICC.2018.8422586 · DOI · Paper page
  55. 10.1109/TWC.2017.2754494 · DOI
  56. wang, 2018, Towards cooperation in sequential prisoner’s dilemmas: A deep multiagent reinforcement learning approach, arXiv preprint arXiv 1803 00162, 1
  57. arnold, 2016, Deep reinforcement learning in large discrete action spaces, arXiv 1512 07679, 1
  58. 10.14778/3199517.3199521 · DOI
  59. 10.1109/MNET.2018.1800085 · doi:10.1109/mnet.2018.1800085
  60. zhao, 2018, Deep reinforcement learning for network slicing, arXiv preprint arXiv 1805 06591, 1
  61. 2016, Google Achieves Ai “Breakthrough” by Beating Go Champion
  62. bellman, 2013, Dynamic Programming
  63. bertsekas, 2005, Dynamic Programming and Optimal Control, 1
  64. 10.1109/MSP.2017.2743240 · DOI · Paper page
  65. 10.1109/JIOT.2017.2759728 · DOI · Paper page
  66. 10.1109/MCOM.2017.1600951 · DOI · Paper page
  67. 10.1109/ACCESS.2018.2836950 · DOI · Paper page
  68. chen, 2018, Reinforcement learning based QoS/QoEaware service function chaining in software-driven 5G slices, Trans Emerg Telecommun Technol, 29
  69. 10.1109/T-WC.2008.071349 · DOI · Paper page
  70. 10.1109/TCCN.2018.2809722 · DOI · Paper page
  71. govindan, 0, Tutornet A Low Power Wireless IoT Testbed
  72. 10.1109/TWC.2012.121112.120018 · DOI · Paper page
  73. heinrich, 2016, Deep reinforcement learning from self-play in imperfect-information games, arXiv preprint arXiv 1603 01840, 1
  74. wang, 2017, Deep reinforcement learning for dynamic multichannel access, Proc Int Conf Comput Netw Commun (ICNC), 599
  75. 10.1109/JSAC.2015.2417011 · DOI · Paper page
  76. 10.1109/WCSP.2018.8555916 · DOI
  77. 2016, Measuring Fixed Broadband Report
  78. 10.1145/3240508.3240545 · DOI
  79. 10.1109/72.329697 · DOI · Paper page
  80. ferreir, 2018, Multiobjective reinforcement learning for cognitive satellite communications using deep neural network ensembles, IEEE J Sel Areas Commun, 36, 1030, 10.1109/JSAC.2018.2832820 · DOI
  81. 10.1109/ICC.2013.6655277 · DOI
  82. lillicrap, 2016, Continuous control with deep reinforcement learning, CoRR, abs 1509 2971, 1
  83. dulac-arnold, 2016, Reinforcement learning in large discrete action spaces, CoRR, abs 1512 7679, 1
  84. 10.1142/9781848162778_0008 · DOI · Paper page
  85. 10.1007/11840817_86 · DOI
  86. 0, Tyouku of China Network Video Index
  87. stockhammer, 2011, Dynamic adaptive streaming over HTTP: Standards and design principles, Proc 2nd Annu ACM Conf Multimedia Syst, 133, 10.1145/1943552.1943572 · DOI · Paper page
  88. 10.1109/TCCN.2017.2755007 · DOI · Paper page
  89. 10.1007/978-3-540-45232-4_16 · DOI · Paper page
  90. 10.1145/3098822.3098843 · DOI · Paper page
  91. 10.1145/2483977.2483991 · DOI · Paper page
  92. 10.1145/2829988.2787486 · DOI · Paper page
  93. 10.1109/ICDCS.2017.21 · DOI
  94. 10.1109/IntelliSys.2017.8324237 · DOI
  95. 10.1109/COMST.2018.2870996 · DOI
  96. 10.1109/LCOMM.2017.2776917 · DOI
  97. zhan, 0, Crowdsensing game with demand uncertainties A deep reinforcement learning approach
  98. 10.1109/COMST.2017.2647981 · DOI · Paper page
  99. 10.1109/COMST.2016.2582841 · DOI · Paper page
  100. 10.1109/TVT.2018.2867191 · DOI · Paper page
  101. 10.1109/TVT.2017.2760281 · DOI · Paper page
  102. 10.1145/3132340.3132355 · DOI
  103. 10.1109/TIT.2011.2173241 · DOI · Paper page
  104. 10.1109/VTCFall.2017.8288203 · DOI
  105. bai, 2018, Deep learning based fast multiuser detection for massive machine-type communication, arXiv preprint arXiv 1807 00967, 1
  106. 10.1109/GLOCOM.2017.8254717 · DOI
  107. chen, 2017, Machine learning for wireless networks with artificial intelligence: A tutorial on neural networks, CoRR, abs 1710 2913, 1
  108. 10.1109/MWC.2018.1700274 · DOI
  109. he, 0, Trust-based social networks with computing, caching and communications: A deep reinforcement learning approach, IEEE Transactions on Network Science and Engineering
  110. 10.1109/MCOM.2017.1700246 · DOI · Paper page
  111. 10.1109/MCOM.2015.7045396 · DOI · Paper page
  112. 10.1145/1807128.1807152 · DOI · Paper page
  113. schaarschmidt, 2016, Learning runtime parameters in computer systems with delayed experience injection, arXiv preprint arXiv 1610 09975, 1
  114. he, 2017, Cache-enabled wireless networks with opportunistic interference alignment, arXiv preprint arXiv 1706 09024, 1
  115. 10.1109/SPAWC.2015.7227119 · DOI
  116. 10.1109/ISWCS.2017.8108157 · DOI
  117. chen, 2018, Echo-liquid state deep learning for content transmission and caching in wireless VR networks with cellular-connected UAVs, arXiv preprint arXiv 1804 00209, 1
  118. 10.1109/ICC.2017.7996332 · DOI
  119. 10.1109/TVT.2017.2751641 · DOI · Paper page
  120. 10.1109/TETC.2018.2805718 · DOI · Paper page
  121. 10.1145/2491224.2491240 · DOI
  122. 10.1145/3219819.3219918 · DOI
  123. 10.1587/transcom.2017CQP0014 · DOI
  124. 10.1587/transcom.2017NRP0014 · doi:10.1587/transcom.2017nrp0014
  125. 10.1016/S0004-3702(02)00121-2 · DOI · Paper page
  126. 10.1109/TVT.2017.2785414 · DOI · Paper page
  127. 10.1109/GLOCOM.2017.8253988 · DOI
  128. 10.1109/TVT.2018.2789466 · DOI · Paper page
  129. 10.1109/ITSC.2018.8569635 · DOI
  130. 10.1109/TAC.2017.2741778 · DOI
  131. 10.1016/S1369-8478(00)00005-X · DOI · Paper page
  132. 10.1109/CEC.2000.870304 · DOI · Paper page
  133. 10.1109/ICC.2018.8422728 · DOI
  134. 10.1109/JPROC.2016.2575064 · DOI · Paper page
  135. 10.1109/CISP-BMEI.2017.8302157 · DOI
  136. 10.1145/3163080.3163113 · DOI
  137. 10.1109/GlobalSIP.2017.8309082 · DOI
  138. 10.1109/TCNS.2014.2378876 · DOI
  139. 10.1109/JSAC.2016.2612038 · DOI
  140. faris, 2018, Deep Q-learning for self-organizing networks fault management and radio performance improvement
  141. stampa, 2017, A deep-reinforcement learning approach for software-defined networking routing optimization, arXiv preprint arXiv 1709 02505, 1
  142. roughan, 2015, Simplifying the synthesis of Internet traffic matrices, ACM SIGCOMM Comput Commun Rev, 35, 93, 10.1145/1096536.1096551 · DOI · Paper page
  143. 10.4108/ICST.SIMUTOOLS2008.3027 · DOI · Paper page
  144. goodfellow, 2016, Deep Learning
  145. 10.1109/INFOCOM.2018.8485853 · DOI · Paper page
  146. 10.1145/2486001.2486020 · DOI · Paper page
  147. 10.1109/MWC.2018.1700291 · DOI · Paper page
  148. 10.1109/GLOCOM.2017.8254503 · DOI
  149. 10.1109/JIOT.2018.2846694 · DOI
  150. 10.1109/ICCW.2018.8403505 · DOI
  151. ye, 2018, DRAG: Deep reinforcement learning based base station activation in heterogeneous networks, arXiv 1809 02159, 1
  152. 10.1109/JIOT.2018.2876279 · DOI · Paper page
  153. 10.1109/VTCFall.2018.8690980 · DOI · Paper page
  154. 10.1109/INFCOMW.2015.7179384 · DOI
  155. 10.1109/ICPR.2010.1057 · DOI
  156. 10.1109/COMST.2018.2846401 · DOI · Paper page
  157. chen, 2017, Machine learning for wireless networks with artificial intelligence: A tutorial on neural networks, arXiv preprint arXiv 1710 02913, 1
  158. 10.1109/COMST.2016.2539923 · DOI
  159. lowe, 2017, Multi-agent actor-critic for mixed cooperative-competitive environments, Proc Adv Neural Inf Process Syst, 6379
  160. 10.1287/mnsc.28.1.1 · DOI · Paper page
  161. 10.1073/pnas.39.10.1953 · DOI · Paper page
  162. 10.1109/MWC.2006.275200 · DOI
  163. hu, 2003, Nash Q-learning for general-sum stochastic games, J Mach Learn Res, 4, 1039
  164. 10.1109/TSC.2018.2827070 · DOI · Paper page
  165. 10.1109/ICASSP.2017.7952524 · DOI · Paper page
  166. 10.1109/TVT.2018.2852330 · DOI
  167. 10.3390/fi10070060 · DOI
  168. 10.1109/PIMRC.2017.8292514 · DOI · Paper page
  169. le, 2018, A deep reinforcement learning based offloading scheme in ad-hoc mobile clouds, Proc IEEE INFOCOM IECCO Workshop, 760
  170. 10.1109/LCOMM.2018.2792015 · DOI
  171. 10.1109/LCOMM.2018.2815018 · DOI · Paper page
  172. 10.1109/VTCSpring.2018.8417695 · DOI
  173. lu, 2019, UAV-aided cellular communications with deep reinforcement learning against jamming, arXiv preprint arXiv 1805 06628, 1

Cited by

See all 2,060

Metadata

Authors
Luong, N. C., Hoang, D. T., Gong, S., Niyato, D., Wang, P., Liang, Y.-C., & Kim, D. I.
Venue
IEEE Communications Surveys & Tutorials
Publisher
Institute of Electrical and Electronics Engineers (IEEE)
Published
2019
Volume
21
Issue
4
Pages
3133-3174
ISSN
  • 1553-877X (ISSN-L)
  • 2373-745X
DOI
10.1109/comst.2019.2916583
OpenAlex
W2898035736
Cited by
2,060
References
27
FWCI
104.64
Type
journal article
Language
en
Open access
closed
Licence
Funders
  • A*STAR-NTU-SUTD Joint Research Grant Call on Artificial Intelligence for the Future of Manufacturing (RGANS1906)
  • WASP/NTU (M4082187 (4080))
  • Ministry of Education - Singapore (2017-T1-002-007 RG122/17)
  • MOE Tier 2 (MOE2014-T2-2-015 ARC4/15)
  • Singapore (NRF2015-NRF-ISF001-2277)
  • Singapore EMA Energy Resilience (NRF2017EWT-EP003-041)
  • National Research Foundation of Korea
  • Ministry of Science, ICT and Future Planning (2014R1A5A1011478, 2017R1A2B2003953)
  • National Natural Science Foundation of China (61631005, U1801261, 61571100, 61601449)
  • Shenzhen Talent Peacock Plan Program (KQTD2015071715073798)

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Applications of Deep Reinforcement Learning in Communications and Networking: A Survey. https://app.cochat.ai/papers/10.1109/comst.2019.2916583
The paper
Luong, N. C., Hoang, D. T., Gong, S., Niyato, D., Wang, P., Liang, Y.-C., & Kim, D. I. (2019). Applications of Deep Reinforcement Learning in Communications and Networking: A Survey. IEEE Communications Surveys & Tutorials. https://doi.org/10.1109/comst.2019.2916583

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.