Collaborative Distributed Q-Learning for RACH Congestion Minimization in Cellular IoT Networks
Open accessThe DOI matches its Crossref record; 15 of the 16 references it cites were found in a registry or an index.
Abstract
Due to infrequent and massive concurrent access requests from the ever-increasing number of machine-type communication (MTC) devices, the existing contention-based random access (RA) protocols, such as slotted ALOHA, suffer from the severe problem of random access channel (RACH) congestion in emerging cellular IoT networks. To address this issue, we propose a novel collaborative distributed Q-learning mechanism for the resource-constrained MTC devices in order to enable them to find unique RA slots for their transmissions so that the number of possible collisions can be significantly reduced. In contrast to the independent Q-learning scheme, the proposed approach utilizes the congestion level of RA slots as the global cost during the learning process and thus can notably lower the learning time for the low-end MTC devices. Our results show that the proposed learning scheme can significantly minimize the RACH congestion in cellular IoT networks.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“For example, ML techniques can be significantly advantageous in addressing the access congestion problem in emerging ultra-dense IoT networks [14].”
From Quantum Machine Learning for 6G Communication Networks: State-of-the-Art and Vision for the Future (2019) · cites this paper as [14]
Citations
Cites
Watkins, C. J. C. H., & Dayan, P. (1992) · Machine Learning · doi:10.1007/bf00992698 · Cited by 9,083
Watkins, C. J. C. H., & Dayan, P. (1992) · Machine Learning · doi:10.1023/a:1022676722315 · Cited by 3,907 · cited 2×
“Among several ML techniques, Q-learning is model-free, computationally simpler and can be implemented in a distributed way [10], [12], thus being suitable for mMTC devices.”
A Comprehensive Survey of Multiagent Reinforcement Learning
Busoniu, L., Babuska, R., & De Schutter, B. (2008) · IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews) · doi:10.1109/tsmcc.2007.913919 · Cited by 2,235 · cited 2×
“This is known as a Q-learning process and the Q-value at each state is calculated by using the following iterative procedure [10], [13].”
MethodToward Massive, Ultrareliable, and Low-Latency Wireless Communication With Short Packets
Durisi, G., Koch, T., & Popovski, P. (2016) · Proceedings of the IEEE · doi:10.1109/jproc.2016.2537298 · Cited by 1,154
Massive machine-type communications in 5g: physical and MAC-layer solutions
Bockelmann, C., Pratas, N., Nikopour, H., Au, K., Svensson, T., Stefanovic, C., Popovski, P., & Dekorsy, A. (2016) · IEEE Communications Magazine · doi:10.1109/mcom.2016.7565189 · Cited by 748 · cited 2×
“Moreover, several novel RA schemes such as coded random access and sparse code multiple access based on compressive sensing-based multiuser detection are being considered for the MTC systems [1].”
BackgroundIntelligent 5G: When Cellular Networks Meet Artificial Intelligence
Li, R., Zhao, Z., Zhou, X., Ding, G., Chen, Y., Wang, Z., & Zhang, H. (2017) · IEEE Wireless Communications · doi:10.1109/mwc.2017.1600304wc · Cited by 601 · cited 1×
“In this regard, emerging Machine Learning (ML)-assisted techniques seem promising since they can play significant roles in learning system variations/parameter uncertainties and adapting system parameters accordingly [8].”
BackgroundLaya, A., Alonso, L., & Alonso-Zarate, J. (2014) · IEEE Communications Surveys & Tutorials · doi:10.1109/surv.2013.111313.00244 · Cited by 574 · cited 1×
“On the other hand, the existing contention-based protocols are highly inefficient in supporting massive MTC devices [5], thus leading to the need of efficient access protocols.”
BackgroundLTE/LTE-A Random Access for Massive Machine-Type Communications in Smart Cities
Ali, M. S., Hossain, E., & Kim, D. I. (2017) · IEEE Communications Magazine · doi:10.1109/mcom.2017.1600215cm · Cited by 167 · cited 1×
“Besides, some other schemes such as prioritized RA, grouped-based RA and code-expanded RA have been studied in the literature [7].”
BackgroundGhavamzadeh, M., Kappen, H. J., Azar, M. G., & Munos, R. (2011) · Radboud Repository (Radboud University) · Cited by 147
Narrowband IoT Data Transmission Procedures for Massive Machine-Type Communications
Andres-Maldonado, P., Ameigeiras, P., Prados-Garzon, J., Navarro-Ortiz, J., & Lopez-Soler, J. M. (2017) · IEEE Network · doi:10.1109/mnet.2017.1700081 · Cited by 128 · cited 1×
“…networks are optimized for the traditional human-type traffic, the incorporation of massive MTC (mMTC) devices will lead to several challenges including Radio Access Network (RAN) congestion, Quality of Service (QoS) provisioning, ultra-low device complexity and limited battery lifetime [2], [3].”
Background
Cited by
Satellite Communications in the New Space Era: A Survey and Future Challenges
Kodheli, O., Lagunas, E., Maturo, N., Sharma, S. K., Shankar, B., Montoya, J. F. M., Duncan, J. C. M., Spano, D., Chatzinotas, S., Kisseleff, S., et al. (2021) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2020.3028247 · Cited by 1,524
“In contrast to these approaches which require training data-sets, the RL does not need a training data-set and enables a learning agent to learn from the prior experience, and seems more suitable for dynamic wireless environments [335], [336].”
BackgroundSelf-citationQuantum Machine Learning for 6G Communication Networks: State-of-the-Art and Vision for the Future
Nawaz, S. J., Sharma, S. K., Wyne, S., Patwary, M. N., & Asaduzzaman, M. (2019) · IEEE Access · doi:10.1109/access.2019.2909490 · Cited by 658
“For example, ML techniques can be significantly advantageous in addressing the access congestion problem in emerging ultra-dense IoT networks [14].”
BackgroundSelf-citationCellular, Wide-Area, and Non-Terrestrial IoT: A Survey on 5G Advances and the Road Toward 6G
Vaezi, M., Azari, A., Khosravirad, S. R., Shirvanimoghaddam, M., Azari, M. M., Chasaki, D., & Popovski, P. (2022) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2022.3151028 · Cited by 631
Sharma, S. K., & Wang, X. (2020) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2019.2916177 · Cited by 427
“At each instance of transmission, the node selects a slot with the highest Q value and in case of two or more maximum values, a random selection approach can be applied [188].”
BackgroundMethodSelf-citationHussain, F., Hassan, S. A., Hussain, R., & Hossain, E. (2020) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2020.2964534 · Cited by 395
“Collaborative distributed Q-learning was proposed in [94] to reduce congestion in the access channel for resourceconstrained MTC devices in IoT networks.”
BackgroundToward Tactile Internet in Beyond 5G Era: Recent Advances, Current Issues, and Future Directions
Sharma, S. K., Woungang, I., Anpalagan, A., & Chatzinotas, S. (2020) · IEEE Access · doi:10.1109/access.2020.2980369 · Cited by 190
“Furthermore, congestion and system overloading are other crucial issues since the concurrent transmissions from a huge number of M2M devices may create congestion in both the random access and traffic channels [114], [193].”
BackgroundSelf-citationSalh, A., Audah, L., Shah, N. S. M., Alhammadi, A., Abdullah, Q., Kim, Y. H., Al-Gailani, S. A., Hamzah, S. A., Esmail, B. A. F., & Almohammedi, A. A. (2021) · IEEE Access · doi:10.1109/access.2021.3069707 · Cited by 181
“Furthermore, based on the high number of collisions in the IoT access network, the developing ultra-dense IoT networks are able to address ML to enable devices to regularly learn the random-access channel in cellular IoT networks [23]–[25].”
BackgroundEdge QoE: Computation Offloading With Deep Reinforcement Learning for Internet of Things
Lu, H., He, X., Du, M., Ruan, X., Sun, Y., & Wang, K. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.2981557 · Cited by 176
From 5G to 6G Technology: Meets Energy, Internet-of-Things and Machine Learning: A Survey
Mahdi, M. N., Ahmad, A. R., Qassim, Q. S., Natiq, H., Subhi, M. A., & Mahmoud, M. A. (2021) · Applied Sciences · doi:10.3390/app11178117 · Cited by 133
“Two papers [169,170] presented a collaborative distributed Q-learning approach for resource-constrained MTC devices to enable them to locate unique RA slots for their transmissions, hence reducing the number of possible collisions.”
BackgroundDeep Reinforcement Learning for Throughput Improvement of the Uplink Grant-Free NOMA System
Zhang, J., Tao, X., Wu, H., Zhang, N., & Zhang, X. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.2972274 · Cited by 133
“With the expansion of network scale, deep RL (DRL) can help improve the efficiency of RL, and therefore, has been adopted for the throughput improvement in grant-free OMA systems [5]–[8].”
BackgroundMethodLearning-Driven Wireless Communications, towards 6G
Piran, M. J., & Suh, D. Y. (2019) · 2019 International Conference on Computing, Electronics & Communications Engineering (iCCECE) · doi:10.1109/iccece46942.2019.8941882 · Cited by 61
A NOMA-Based Q-Learning Random Access Method for Machine Type Communications
da Silva, M. V., Souza, R. D., Alves, H., & Abrão, T. (2020) · IEEE Wireless Communications Letters · doi:10.1109/lwc.2020.3002691 · Cited by 74
“The third method is Collaborative Q -Learning [10].”
BackgroundMethodInfluentialArtificial intelligence implication on energy sustainability in Internet of Things: A survey
Charef, N., Mnaouer, A. B., Aloqaily, M., Bouachir, O., & Guizani, M. (2022) · Information Processing & Management · doi:10.1016/j.ipm.2022.103212 · Cited by 66
Distributed Learning for Wireless Communications: Methods, Applications and Challenges
Qian, L., Yang, P., Xiao, M., Dobre, O. A., Renzo, M. D., Li, J., Han, Z., Qin, Y., & Zhao, J. (2022) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2022.3156756 · Cited by 60
Gao, A., Du, C., Ng, S. X., & Liang, W. J. (2021) · IEEE Communications Letters · doi:10.1109/lcomm.2021.3078442 · Cited by 59
“On this basis, SUs can find the “vacant position” cooperatively and occupy the unused spectrum bands to transmit information temporarily by only partial observation of the frequency band [2], [3], [4].”
BackgroundDistributed Q-Learning Aided Uplink Grant-Free NOMA for Massive Machine-Type Communications
Liu, J., Shi, Z., Zhang, S., & Kato, N. (2021) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2021.3078496 · Cited by 53
“It is worth noting that all of the above RL-assisted schemes [14]–[17] assume that the device has saturated traffic (i.e., the device essentially always has data to be transmitted).”
BackgroundMethodResultInfluentialDas, B. R., Hasan, S. R., Sabuj, S. R., Hossain, M. A., & Ray, S. K. (2025) · International Journal of Intelligent Networks · doi:10.1016/j.ijin.2025.06.001 · Cited by 49
Jiang, N., Deng, Y., & Nallanathan, A. (2021) · IEEE Communications Magazine · doi:10.1109/mcom.001.2000099 · Cited by 46
“2) Dynamic Resource Allocation (DRA): BS allocates a number of channels for RACH according to the requirements during congestion [6].”
BackgroundMethodInfluentialTran, D., Sharma, S. K., Ha, V. N., Chatzinotas, S., & Woungang, I. (2023) · IEEE Open Journal of the Communications Society · doi:10.1109/ojcoms.2023.3291689 · Cited by 45
“Intelligent features are an important aspect of future cellular networks, and many current research works have applied RL-based algorithms to address the collisions and severe interference in massive access scenarios [20], [21], [22], [23], [24], [25], [26], [27], [28], [29], [30], [31].”
BackgroundSelf-citationGao, A., Wang, Q., Wang, Y., Du, C., Hu, Y., Liang, W. J., & Ng, S. X. (2024) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2024.3384393 · Cited by 42
“Thanks to the non-linear approximation ability of deep neural networks [21], the model-free feature of DRL, such as Q-learning [11], DQN [10], and DDPG [22], makes it easier to map the high dimensional state space to a specific action without the convexity requirement for objective or reward…”
BackgroundAI enabled value-oriented collaborative learning: Centre for innovative education
Ramadevi, J., Sushama, C., Balaji, K., Talasila, V., Sindhwani, N., & Mukti, M. (2023) · The Journal of High Technology Management Research · doi:10.1016/j.hitech.2023.100478 · Cited by 40
Reinforcement learning algorithm for 5G indoor device‐to‐device communications
Sreedevi, A. G., & Rao, T. R. (2019) · Transactions on Emerging Telecommunications Technologies · doi:10.1002/ett.3670 · Cited by 39
Subcarrier Assignment Schemes Based on Q-Learning in Wideband Cognitive Radio Networks
Zhou, Y., Zhou, F., Wu, Y., Hu, R. Q., & Wang, Y. (2019) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2019.2953809 · Cited by 37
“5 shows the number of the successfully accessing SUs versus the number of iterations achieved with our proposed two Q-learning based methods and that obtained with the collaborative and independent Q-learning methods in [19].”
BackgroundMethodInfluentialA Survey on NB-IoT Random Access: Approaches for Uplink Radio Access Network Congestion Management
Iiyambo, L., Hancke, G. P., & Abu-Mahfouz, A. M. I. (2024) · IEEE Access · doi:10.1109/access.2024.3419216 · Cited by 36
“Reference [88] and [89] address RAN congestion by proposing Q-learning techniques to determine unique time slots for their transmissions.”
BackgroundMethodZhao, B. W., Ren, G., Dong, X., & Zhang, H. (2021) · IEEE Communications Letters · doi:10.1109/lcomm.2021.3061717 · Cited by 31
“These active devices are assumed to be backlogged, i.e., devices always have packets to transmit, as in [11].”
BackgroundUpreti, K., Kumar, N., Alam, M. S., Verma, A., Nandan, M., & Gupta, A. K. (2021) · International Conference on Electrical, Computer and Communication Technologies (ICECCT) · doi:10.1109/icecct52121.2021.9616864 · Cited by 31
Optimal User Pairing and Power Allocation in 5G Satellite Random Access Networks
Zhao, B. W., Dong, X., Ren, G., & Liu, J. (2021) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2021.3126579 · Cited by 29
“In addition, intelligent algorithms (such as Q-learning) have been proven to be effective solutions, and have been used in many wireless networks, such as cellular networks, Ad-Hoc networks [30]–[32].”
MethodMulti-Agent Reinforcement Learning Based Uplink OFDMA for IEEE 802.11ax Networks
Han, M., Sun, X., Zhan, W., Gao, Y., & Jiang, Y. (2024) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2024.3355276 · Cited by 25
“…(MARL) technique has been widely adopted in recent literature, which allows devices to learn access strategies cooperatively to avoid collisions between devices and improve the throughput while guaranteeing fairness [4], [5], [6], [7], [8], [9], [10], [11], [12], [13], [14], [15], [16], [17].”
MethodBLER-based Adaptive Q-learning for Efficient Random Access in NOMA-based mMTC Networks
Tran, D., Sharma, S. K., & Chatzinotas, S. (2021) · IEEE Vehicular Technology Conference · doi:10.1109/vtc2021-spring51267.2021.9448787 · Cited by 21
Self-citationRen, Z., Jin, J., Li, W., & Zhan, Y. (2021) · IEEE Transactions on Aerospace and Electronic Systems · doi:10.1109/taes.2021.3130832 · Cited by 20
Umar, A., Khalid, Z., Ali, M., Abazeed, M. B., Alqahtani, A., Ullah, R., & Safdar, H. (2023) · Applied Sciences · doi:10.3390/app132212384 · Cited by 19
Distributed Q-Learning-Assisted Grant-Free NORA for Massive Machine-Type Communications
Shi, Z., Gao, W., Liu, J., Kato, N., & Zhang, Y. (2020) · Globecom · doi:10.1109/globecom42002.2020.9322273 · Cited by 19
“Specifically, a distributed Q-learning algorithm was adopted in [9] to tackle the challenges of massive MTC devices to traditional centralized scheduling policies.”
BackgroundMethodD2D Assisted Q-Learning Random Access for NOMA-Based MTC Networks
da Silva, M. V., Montejo‐Sánchez, S., Souza, R. D., Alves, H., & Abrão, T. (2022) · IEEE Access · doi:10.1109/access.2022.3160156 · Cited by 19
“Our work differs from [19], [20], [27] because, besides usingQ-Learning for resource allocation purposes, we implement NOMA for improving spectral efficiency.”
BackgroundMethodInfluentialZhan, W., Sun, X., Wang, X., Fu, Y., & Li, Y. (2021) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2021.3096259 · Cited by 18
Self-citationTran, D., Ha, V. N., & Chatzinotas, S. (2022) · IEEE Vehicular Technology Conference · doi:10.1109/vtc2022-spring54318.2022.9860574 · Cited by 18
“This work investigates both original and admission-limited GF-NOMA communication schemes which are named as: • Without SDC (woSDC) : All K SCs are available for all devices to compete for their transmissions [10–13].”
BackgroundMethodA Systematic Review of LPWAN and Short-Range Network using AI to Enhance Internet of Things
Widianto, M. H., Sinaga, A., & Ginting, M. A. (2022) · Journal of Robotics and Control (JRC) · doi:10.18196/jrc.v3i4.15419 · Cited by 17
Fairness aware deep reinforcement learning for grant-free NOMA-IoT networks
Balcı, A., & SOKULLU, R. İ. (2024) · Internet of Things · doi:10.1016/j.iot.2024.101079 · Cited by 17
Diffusion Bayesian Subband Adaptive Filters for Distributed Estimation Over Sensor Networks
Huang, F., Zhang, J., Zhang, S., Chen, H., & Poor, H. V. (2021) · IEEE Transactions on Communications · doi:10.1109/tcomm.2021.3100624 · Cited by 17
Reinforcement-Learning-Aided NOMA Random Access: An AoI-Based Timeliness Perspective
Pereira, F. M., Farhat, J., Rebelatto, J. L., Brante, G. G. D. O., & Souza, R. D. (2024) · IEEE Internet of Things Journal · doi:10.1109/jiot.2024.3524177 · Cited by 16
“3 refers to the scheme from [13], where the Q -learning reward depends on the number of colliding devices.”
Distributive Throughput Optimization for Massive Random Access of M2M Communications in LTE Networks
Zhang, C., Sun, X., Zhang, J., Wang, X., Jin, S., & Zhu, H. (2020) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2020.3019814 · Cited by 16
“The aforementioned works, either aim to choose conflict-free time slots [22], [23] or satisfy the requirements of different”
BackgroundSelf-citationMassive connectivity with machine learning for the Internet of Things
Balcı, A., & SOKULLU, R. İ. (2020) · Computer Networks · doi:10.1016/j.comnet.2020.107646 · Cited by 16
Zhu, B., & Wang, X. (2025) · IEEE Transactions on Network Science and Engineering · doi:10.1109/tnse.2025.3592859 · Cited by 15
“To address the challenge, this paper proposes a networked physical computing system that integrates the physical attributes of computing resources and tasks as well as task-specific trust relationships among devices to enable value-driven task completion.”
Self-citationDesign and Analysis of a Dynamic Access Class Barring NOMA Random Access Algorithm
Hu, Y., Lin, P., & Liu, Y. (2022) · IEEE Communications Letters · doi:10.1109/lcomm.2022.3204567 · Cited by 15
“By the method of continuous information exchange between the device and a base station (BS), a collaborative distributed Q-learning algorithm is presented to help every device to find a RA slot in [2].”
MethodThroughput and latency in the distributed Q-learning random access mMTC networks
Silva, G. M. F., & Abrão, T. (2022) · Computer Networks · doi:10.1016/j.comnet.2022.108787 · Cited by 15
Q-Learning-Based SCMA for Efficient Random Access in mMTC Networks With Short Packets
Tran, D., Sharma, S. K., Chatzinotas, S., & Woungang, I. (2021) · IEEE International Symposium on Personal, Indoor and Mobile Radio Communications (PIMRC) · doi:10.1109/pimrc50174.2021.9569713 · Cited by 14
“Q-learning is one of the most popular reinforcement learning algorithms, which enables agents to learn from the previous experiment in order to achieve successful strategies by inter-acting with the environment [10].”
BackgroundMethodInfluentialSelf-citationAn SCMA-Based Decoupled Distributed Q-Learning Random Access Scheme for Machine-Type Communication
Su, J., Ren, G., Wang, Q., & Zhao, B. W. (2021) · IEEE Wireless Communications Letters · doi:10.1109/lwc.2021.3078476 · Cited by 13
Stochastic Multi-Player Multi-Armed Bandits with Multiple Plays for Uncoordinated Spectrum Access
Youssef, M.-J., Veeravalli, V. V., Farah, J., & Nour, C. A. (2020) · IEEE International Symposium on Personal, Indoor and Mobile Radio Communications (PIMRC) · doi:10.1109/pimrc48278.2020.9217349 · Cited by 12
SCMA-Q-learning with overload control for random access in LEO satellite mMTC networks
Wu, Z., Jing, G., Ding, J., & Zhao, X. (2024) · Physical Communication · doi:10.1016/j.phycom.2024.102584 · Cited by 11
Reddy, Y. S., Dubey, A., Kumar, A., & Panigrahi, T. (2021) · IEEE Access · doi:10.1109/access.2021.3049439 · Cited by 11
“The Q-learning approach in [19] minimizes the congestion level of RA slots as a cost function to enhance the success rate.”
BackgroundGenerative Artificial Intelligence and the 4C Learning Skills for the 21st Century
Türker, P. M., & Öztürk, M. (2024) · Advances in educational technologies and instructional design book series · doi:10.4018/979-8-3693-7220-3.ch011 · Cited by 10
Priority-Aware Access Strategy for GF-NOMA System in IIoT: The Device-Specific Allocation Approach
Wu, D., Zhang, Z., Huang, Y., & Qin, X. (2023) · IEEE Internet of Things Journal · doi:10.1109/jiot.2023.3292894 · Cited by 10
“In [31], a collaborative distributed Q -learning scheme with the congestion level of RA slots as the reward was proposed for MTC devices to select the suitable RA slots. da Silva et al. [32] presented a distributed Q -learning method based on NOMA to dynamically allocate RA slots.”
MethodDistributed Multi-Agent Reinforcement Learning for Heterogeneous NOMA-ALOHA Systems
Wu, X., Ko, Y., & Tyrrell, A. M. (2024) · IEEE Transactions on Cognitive Communications and Networking · doi:10.1109/tccn.2024.3474709 · Cited by 9
“Moreover, the congestion levels used in [18] and [19] are non-binary numbers requiring control link from BS to users, which is a resource cost.”
Lee, H., Kim, E., Kim, H., Na, J.-H., & Choi, H.-H. (2021) · ICT Express · doi:10.1016/j.icte.2021.09.006 · Cited by 9
Silva, G. M. F., & Abrão, T. (2022) · Transactions on Emerging Telecommunications Technologies · doi:10.1002/ett.4509 · Cited by 9
“The devices only send all their UL packets and the number of successes is obtained when there is no collision; b) Independent QL [19]; c) Collaborative QL [19]; and Packet-Based QL [21].”
BackgroundMethodInfluentialLearning-Based Multiplexing of Grant-Based and Grant-Free Heterogeneous Services with Short Packets
Tran, D., Sharma, S. K., Chatzinotas, S., & Woungang, I. (2021) · Globecom · doi:10.1109/globecom46510.2021.9685321 · Cited by 9
“In the recent years, reinforcement learning (RL)-based smart CC method has drawn an important attention [1, 6–11].”
BackgroundMethodResultInfluentialSelf-citationRACH Traffic Prediction in Massive Machine Type Communications
Mehri, H., Mehrpouyan, H., & Chen, H. (2025) · IEEE Transactions on Machine Learning in Communications and Networking · doi:10.1109/tmlcn.2025.3542760 · Cited by 8
“Most of the existing methods of predicting the traffic pattern focus on human type traffic or ignores the event-driven bursty traffic [31], [43], [46], [48]–[50].”
An Access Control Scheme Combining Q-Learning and Compressive Random Access for Satellite IoT
Jiang, F., Ma, S., Yin, T.-Y., Wang, Y., & Hu, Y. (2023) · IEEE Communications Letters · doi:10.1109/lcomm.2023.3323387 · Cited by 8
“Q-learning is model-free and computationally simpler, thus being suitable for MTC devices with limited energy [15].”
BackgroundOperating models of Network protocols IoT: Short-range protocols
Elhadi, S., Marzak, A., & Sael, N. (2020) · International Symposium on Advanced Electrical and Communication Technologies (ISAECT) · doi:10.1109/isaect50560.2020.9523646 · Cited by 7
Enhanced RACH optimization in IoT networks: A DQN approach for balancing H2H and M2M communications
Liu, X., Yang, H., Li, S., Liu, Z., & Lian, X. (2024) · Internet of Things · doi:10.1016/j.iot.2024.101433 · Cited by 6
Tian, D. (2023) · Journal of Physics Conference Series · doi:10.1088/1742-6596/2560/1/012036 · Cited by 6
“[14] proposes an algorithm for allocating resources in cellular networks using DRL.”
MethodBueno, F. A. D., Goedtel, A., Abrão, T., & Marinello, J. C. (2022) · Journal of Sensor and Actuator Networks · doi:10.3390/jsan13030030 · Cited by 6
“This model can then be applied for decentralized pilot selection as described in [28,30], in which each device has a Q-table of τ p elements evaluating its experience in selecting the different pilots to transmit data.”
BackgroundMethodInfluentialRandom Access Based on LSTM for Mixed Traffic IoT Networks
dos Santos, H. L., Marinello, J. C., Panazio, C. M., & Abrão, T. (2023) · IEEE Transactions on Network Science and Engineering · doi:10.1109/tnse.2023.3329654 · Cited by 6
“DRA uses channel co-ordination to avoid collisions in RA procedures [17] at the cost of processing in the BS side, being sensitive to the environment dynamic.”
BackgroundMulti-Armed-Bandit-Based Framed Slotted Aloha for Throughput Optimization
Peng, N., & Dai, L. (2024) · IEEE Communications Letters · doi:10.1109/lcomm.2024.3363461 · Cited by 6
“Instead of exploring advanced coding/decoding design, learning-based schemes have been proposed in [14], [15], [16], and [17], where nodes update the transmission probability in each slot [14] or the slot selection strategy [15], [16], [17] based on the outcome of previous transmissions.”
BackgroundOperating Models of Network Protocols IoT: Long-Range Protocols
Elhadi, S., Marzak, A., & Sael, N. (2021) · Lecture notes in networks and systems · doi:10.1007/978-3-030-66840-2_81 · Cited by 4
A Hybrid Collaborative Learning for Age of Information Minimization in Massive Access
Xu, K., Xu, Y., Wang, X., & Wang, X. (2024) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2024.3467255 · Cited by 3
“Without loss of generality, we assume that when a slot contains only one packet, the packet can be consistently decoded successfully by the BS, a premise that aligns with the majority of prior work in this field [13], [29], [30], [31], [32].”
Self-citationDynamic Medium Access in Clustered NOMA IoT Networks based on Reinforcement Learning
Kumar, A., Martínez-Bauset, J., & Li, F. Y. (2024) · IEEE International Symposium on Personal, Indoor and Mobile Radio Communications (PIMRC) · doi:10.1109/pimrc59610.2024.10817308 · Cited by 3
“Furthermore, the study in [7] introduced a QL scheme designed to minimize random access channel collisions by determining the most suitable random access slot for each resource-constrained MTC device.”
Kaur, U., & Shalu, S. (2021) · Springer eBooks · doi:10.1007/978-3-030-67490-8_17 · Cited by 3
Wu, Z. P., Guan, M., Liu, M., Xia, L., Lv, C., Cao, X., & Chen, H. (2021) · Wireless Networks · doi:10.1007/s11276-021-02842-5 · Cited by 3
“Internet of things [1,2] as another wave of information industry after computer and Internet and mobile communication network will continue to show a rapid development trend in the next few years, even decades.”
Device and Network Coordination for Opportunistic Utilization of Radio Resources in 3D Networks
Sriyananda, M. G. S., Wang, X., & Primak, S. L. (2022) · IEEE Open Journal of Vehicular Technology · doi:10.1109/ojvt.2022.3207065 · Cited by 3
Self-citationPreamble slice orderly queue access scheme in cell-free dense communication systems
Sun, J., Guo, M., & Liu, J. (2023) · Digital Communications and Networks · doi:10.1016/j.dcan.2023.05.003 · Cited by 2
Insights on critical energy efficiency approaches in internet-of-things application
Sharath, S. M., Manjunatha, P., & Shwetha, H. R. (2021) · International Journal of Power Electronics and Drive Systems/International Journal of Electrical and Computer Engineering · doi:10.11591/ijece.v11i4.pp2925-2933 · Cited by 2
“Most recently, the problems associated with the traffic bottleneck situation have been addressed by Sharma and Wang [36] with the Q-learning approach.”
BackgroundAn Intelligent Access Channel Algorithm Based on Distributed Double Q Learning
Zhang, G., Niu, Y., Li, Y., & Zhao, L. (2022) · Applied Sciences · doi:10.3390/app122110815 · Cited by 1
Performance Enhancement using Q-Learning and Dynamic Clustering in Internet of Things Applications
Gobinathan, B., Jayamani, K., Sabapathi, R., Jagadeeshwaran, A., Abid, Y., & Liakath, J. A. (2024) · International Conference on Innovative Computing, Intelligent Communication and Smart Electrical Systems (ICSES) · doi:10.1109/icses63760.2024.10910263 · Cited by 1
“In [13], A suggested cooperative distributed Q-learning method improves resource distribution for machine-type communication (MTC) devices within cellular IoT networks by taking into account the congestion status of random access (RA) slots as a unified cost, resulting in a significant decrease in…”
Optimization of Dynamic Frame Length for Random Access in Machine-Type Communication Systems
Sun, J., Jing, G., & Ding, J. (2025) · Electronics · doi:10.3390/electronics14173414 · Cited by 1
Delay-Optimal Random Access: A Learning Framework
Zhang, H., Zhao, X., & Dai, L. (2026) · IEEE Transactions on Communications · doi:10.1109/tcomm.2026.3652495 · Cited by 1
“4 Specifically, various RL-based access schemes have been proposed for network throughput maximization with the collision receiver 5 [39], [40], [41], [42], [43], [44], [45], [46], [47], [48], [49], [50].”
Kim, S., Ahn, J., Lee, M., You, Y., Lee, H., & Yu, H. (2026) · ICT Express · doi:10.1016/j.icte.2026.07.015 · Cited by 1
Grant Free NOMA with Deep Learning for Massive IoT Applications
Balcı, A., & SOKULLU, R. İ. (2023) · International Balkan Conference on Communications and Networking (BalkanCom) · doi:10.1109/balkancom58402.2023.10167912 · Cited by 1
“In [7], the study is extended by adding collaboration between MTC devices.”
BackgroundDynamic Spectrum Allocation for mMTC Devices Using SCMA-Q-Learning with ACB Overload Control
Zhang, H., Li, Z., & Wu, Z. (2024) · Lecture notes in electrical engineering · doi:10.1007/978-981-97-8654-1_36 · Cited by 1
A Multi-Layer Strategy for Reducing Network Congestion and Enhancing Performance in 5G IoT Networks
Dlamini, B. M., Migabo, E., & Kurien, A. M. (2024) · International Multidisciplinary Information Technology and Engineering Conference (IMITEC) · doi:10.1109/imitec60221.2024.10851185 · Cited by 1
“To solve this issue, numerous congestion management techniques based on network parameters such as link state estimates [12],[14] packet losses [8], [15] and explicit congestion notification [16] have been suggested.”
Su, J., Fu, M., Zhen, L., Chu, Z., & Xiao, P. (2024) · IEEE Globecom Workshops (GC Wkshps) · doi:10.1109/gcwkshp64532.2024.11100844 · Cited by 1
“For brevity, we name the work in [12]–[14] Q-SA, DQL-SCMA-SA, and DQL-NOMA-SA, respectively.”
Influential2019 Index IEEE Communications Letters Vol. 23
(2019) · IEEE Communications Letters · doi:10.1109/lcomm.2019.2962286 · Cited by 1
Throughput-Optimal Random Access: A Queueing-Theoretical Analysis for Learning-Based Access DesignPreprint
Zhao, X., & Dai, L. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2504.03178 · Cited by 1
Zone based GLRT for detecting physical random access channel signals in 5G
Tütüncüoğlu, F. (2019) · Bilkent University Institutional Repository (Bilkent University)
Congestion Control Mechanisms for 4G Random Access Channel (RACH): A Survey
Magbo, P. C., Chukwudi, P. C., & Abubakar, A. (2019) · International Journal of Engineering and Applied Sciences (IJEAS) · doi:10.31873/ijeas.6.12.02
“Collaborative Distributive Q-learning scheme for addressing the congestion challenge in cellular IoT was proposed in [14].”
BackgroundApplication of Machine Learning to Multi Antenna Transmission and Machine Type Resource Allocation
Emenonye, D.-R. (2020) · VTechWorks (Virginia Tech)
Learning-Assisted Access Management for Dense 3D Small Cell Networks
Sriyananda, M. G. S., Wang, X., & Primak, S. L. (2021) · Conference proceedings - Canadian Conference on Electrical and Computer Engineering · doi:10.1109/ccece53047.2021.9569043
Self-citationInternet of Robotic Things: A New Architecture and Platform
Vijayalakshmi, V., Vimal, S., & Saravanan, M. (2021) · Wiley eBooks · doi:10.1002/9781119752165.ch1
Decentralized Multi-State Q-Learning for NOMA-ALOHA Systems
Wu, X., Ko, Y., & Tyrrell, A. M. (2024) · IEEE International Symposium on Personal, Indoor and Mobile Radio Communications (PIMRC) · doi:10.1109/pimrc59610.2024.10817213
Rate Grouping Based Non-Orthogonal Random Access Aided by Online Traffic Load Prediction
Su, H., Huang, J., Zhao, M., & Zhu, J. (2024) · International Conference on Wireless Communications and Signal Processing (WCSP) · doi:10.1109/wcsp62071.2024.10827029
Probability-Based SCMA Random Access Scheme for Satellite Machine-Type Communication
Wei, Y., & Song, K. (2025) · International Conference on Electronics, Circuits and Information Engineering (ECIE) · doi:10.1109/ecie65947.2025.11086936
“For instance, [3] introduced the NOMA-SA-QL scheme for slot and power allocation, while the collaborative distributed Q-learning algorithm in [4] helps devices find unique RA slots for the slotted-Aloha (SA) scheme, significantly reducing RACH congestion.”
Design and Optimization of Ultra-Reliable Low-Latency Communications in Beyond 5G Networks
TRAN, D. D. (2023) · Open Repository and Bibliography (University of Luxembourg)
Throughput and latency Q-learning-based random access protocols for mMTC systems
Silva, G. M. F. (2021) · LA Referencia (Red Federada de Repositorios Institucionales de Publicaciones Científicas)
dos Santos, E. J. (2023) · LA Referencia (Red Federada de Repositorios Institucionales de Publicaciones Científicas)
Bueno, F. A. D. (2022) · LA Referencia (Red Federada de Repositorios Institucionales de Publicaciones Científicas)
Online and Safe Multiagent RL for Massive Random Access in Tactical Flying Ad Hoc Networks
Jeon, J., Ahn, J., Lee, M., You, Y., Yu, H., & Lee, H. (2026) · IEEE Internet of Things Journal · doi:10.1109/jiot.2026.3676620
“To address these challenges, reinforcement learning (RL)- based approaches have been explored to enable UAVs to autonomously learn optimal transmission policies through iterative interactions with the environment [11], [12], [13], [14], [15], [16], [17], [18], [19], [20].”
New approaches for resource allocation in future communication networks using NOMA and UAVs
Youssef, M.-J. (2020) · theses.fr (ABES) · doi:10.70675/f65bc021z6b9dz4606z84e0z698ca8ff507d
Nonorthogonal Random Access Control Exploiting Timing-Advance Grouping for Cellular IoT Networks
Jang, H. S., Quek, T. Q. S., & Jin, H. (2026) · IEEE Internet of Things Journal · doi:10.1109/jiot.2026.3710035
Wu, D., Song, W., Li, M., Liu, J., Wu, Y., & Tang, F. (2026) · IEEE Open Journal of the Communications Society · doi:10.1109/ojcoms.2026.3710981
Study and computer simulation of random access protocols for IoT networks in 5G systems and beyond
Barreto, J. A. M. (2022) · Institutional Repository of the Federal Technological University of Paraná (RIUT) (Federal University of Technology – Paraná)
Metadata
- Authors
- Sharma, S. K., & Wang, X.
- Affiliations
- Western University (CA)
- Venue
- IEEE Communications Letters
- Published
- 2019-04
- Volume
- 23
- Issue
- 4
- Pages
- 600-603
- ISSN
- 1089-7798 (ISSN-L)
- 1558-2558
- 2373-7891
- OpenAlex
- W2911232490
- Cited by
- 88
- References
- 16
- Citations by year
- 2026: 4 · 2025: 4 · 2024: 19 · 2023: 9 · 2022: 11 · 2021: 22 · 2020: 12 · 2019: 7
- FWCI
- 4.67
- Type
- journal article
- Language
- en
- Open access
- green
- orbilu.uni.lu
- Licence
- Locations
- IEEE Communications Letters (publishedVersion)
- Open Repository and Bibliography (University of Luxembourg) (submittedVersion, cc-by-nc-sa)
- Topics
- Physical Sciences › Engineering › Electrical and Electronic Engineering › IoT Networks and Protocols
- Physical Sciences › Computer Science › Computer Networks and Communications › Wireless Networks and Protocols
- Physical Sciences › Computer Science › Computer Networks and Communications › IoT and Edge/Fog Computing
- Keywords
- machine-type communication, contention-based random access, slotted ALOHA, collision avoidance, resource-constrained devices
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.