Collaborative Distributed Q-Learning for RACH Congestion Minimization in Cellular IoT Networks
Open accessThe DOI matches its Crossref record; 15 of the 16 references it cites were found in a registry or an index.
Abstract
Due to infrequent and massive concurrent access requests from the ever-increasing number of machine-type communication (MTC) devices, the existing contention-based random access (RA) protocols, such as slotted ALOHA, suffer from the severe problem of random access channel (RACH) congestion in emerging cellular IoT networks. To address this issue, we propose a novel collaborative distributed Q-learning mechanism for the resource-constrained MTC devices in order to enable them to find unique RA slots for their transmissions so that the number of possible collisions can be significantly reduced. In contrast to the independent Q-learning scheme, the proposed approach utilizes the congestion level of RA slots as the global cost during the learning process and thus can notably lower the learning time for the low-end MTC devices. Our results show that the proposed learning scheme can significantly minimize the RACH congestion in cellular IoT networks.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“For example, ML techniques can be significantly advantageous in addressing the access congestion problem in emerging ultra-dense IoT networks [14].”
From Quantum Machine Learning for 6G Communication Networks: State-of-the-Art and Vision for the Future (2019) · cites this paper as [14]
Citations
Cites
Watkins, C. J. C. H., & Dayan, P. (1992) · Machine Learning · doi:10.1007/bf00992698 · Cited by 9,083
Watkins, C. J. C. H., & Dayan, P. (1992) · Machine Learning · doi:10.1023/a:1022676722315 · Cited by 3,907 · cited 2×
“Among several ML techniques, Q-learning is model-free, computationally simpler and can be implemented in a distributed way [10], [12], thus being suitable for mMTC devices.”
A Comprehensive Survey of Multiagent Reinforcement Learning
Busoniu, L., Babuska, R., & De Schutter, B. (2008) · IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews) · doi:10.1109/tsmcc.2007.913919 · Cited by 2,235 · cited 2×
“This is known as a Q-learning process and the Q-value at each state is calculated by using the following iterative procedure [10], [13].”
MethodToward Massive, Ultrareliable, and Low-Latency Wireless Communication With Short Packets
Durisi, G., Koch, T., & Popovski, P. (2016) · Proceedings of the IEEE · doi:10.1109/jproc.2016.2537298 · Cited by 1,154
Massive machine-type communications in 5g: physical and MAC-layer solutions
Bockelmann, C., Pratas, N., Nikopour, H., Au, K., Svensson, T., Stefanovic, C., Popovski, P., & Dekorsy, A. (2016) · IEEE Communications Magazine · doi:10.1109/mcom.2016.7565189 · Cited by 748 · cited 2×
“Moreover, several novel RA schemes such as coded random access and sparse code multiple access based on compressive sensing-based multiuser detection are being considered for the MTC systems [1].”
BackgroundIntelligent 5G: When Cellular Networks Meet Artificial Intelligence
Li, R., Zhao, Z., Zhou, X., Ding, G., Chen, Y., Wang, Z., & Zhang, H. (2017) · IEEE Wireless Communications · doi:10.1109/mwc.2017.1600304wc · Cited by 601 · cited 1×
“In this regard, emerging Machine Learning (ML)-assisted techniques seem promising since they can play significant roles in learning system variations/parameter uncertainties and adapting system parameters accordingly [8].”
BackgroundLaya, A., Alonso, L., & Alonso-Zarate, J. (2014) · IEEE Communications Surveys & Tutorials · doi:10.1109/surv.2013.111313.00244 · Cited by 574 · cited 1×
“On the other hand, the existing contention-based protocols are highly inefficient in supporting massive MTC devices [5], thus leading to the need of efficient access protocols.”
BackgroundLTE/LTE-A Random Access for Massive Machine-Type Communications in Smart Cities
Ali, M. S., Hossain, E., & Kim, D. I. (2017) · IEEE Communications Magazine · doi:10.1109/mcom.2017.1600215cm · Cited by 167 · cited 1×
“Besides, some other schemes such as prioritized RA, grouped-based RA and code-expanded RA have been studied in the literature [7].”
BackgroundGhavamzadeh, M., Kappen, H. J., Azar, M. G., & Munos, R. (2011) · Radboud Repository (Radboud University) · Cited by 147
Narrowband IoT Data Transmission Procedures for Massive Machine-Type Communications
Andres-Maldonado, P., Ameigeiras, P., Prados-Garzon, J., Navarro-Ortiz, J., & Lopez-Soler, J. M. (2017) · IEEE Network · doi:10.1109/mnet.2017.1700081 · Cited by 128 · cited 1×
“…networks are optimized for the traditional human-type traffic, the incorporation of massive MTC (mMTC) devices will lead to several challenges including Radio Access Network (RAN) congestion, Quality of Service (QoS) provisioning, ultra-low device complexity and limited battery lifetime [2], [3].”
BackgroundLearning How to Communicate in the Internet of Things: Finite Resources and Heterogeneity
Park, T., Abuzainab, N., & Saad, W. (2016) · IEEE Access · doi:10.1109/access.2016.2615643 · Cited by 121
Application of Q-Learning for RACH Access to Support M2M Traffic over a Cellular Network
Bello, L. M., Mitchell, P., & Grace, D. (2014) · European Wireless Conference · Cited by 46
Access control of MTC devices using reinforcement learning approach
Moon, J., & Lim, Y. (2017) · International Conference on Information Networking · doi:10.1109/icoin.2017.7899576 · Cited by 26 · cited 4×
“In the above context, some recent works have investigated the application of ML techniques for RAN congestion problem in various settings [9], [10].”
BackgroundMethodInfluentialDistributed Caching Enabled Peak Traffic Reduction in Ultra-Dense IoT Networks
Sharma, S. K., & Wang, X. (2018) · IEEE Communications Letters · doi:10.1109/lcomm.2018.2810211 · Cited by 24 · cited 2×
“may arise due to several reasons [3].”
BackgroundSelf-citationLeveraging Edge Computing through Collaborative Machine Learning
Portelli, K., & Anagnostopoulos, C. B. (2017) · International Conference on Future Internet of Things and Cloud Workshops (FiCloudW) · doi:10.1109/ficloudw.2017.72 · Cited by 22 · cited 1×
“To address this, collaborative learning can be utilized by exploiting the global reward/cost of the collective environment instead of only the individual reward for a single device [14].”
Method
Cited by
Satellite Communications in the New Space Era: A Survey and Future Challenges
Kodheli, O., Lagunas, E., Maturo, N., Sharma, S. K., Shankar, B., Montoya, J. F. M., Duncan, J. C. M., Spano, D., Chatzinotas, S., Kisseleff, S., et al. (2021) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2020.3028247 · Cited by 1,524
“In contrast to these approaches which require training data-sets, the RL does not need a training data-set and enables a learning agent to learn from the prior experience, and seems more suitable for dynamic wireless environments [335], [336].”
BackgroundSelf-citationQuantum Machine Learning for 6G Communication Networks: State-of-the-Art and Vision for the Future
Nawaz, S. J., Sharma, S. K., Wyne, S., Patwary, M. N., & Asaduzzaman, M. (2019) · IEEE Access · doi:10.1109/access.2019.2909490 · Cited by 658
“For example, ML techniques can be significantly advantageous in addressing the access congestion problem in emerging ultra-dense IoT networks [14].”
BackgroundSelf-citationCellular, Wide-Area, and Non-Terrestrial IoT: A Survey on 5G Advances and the Road Toward 6G
Vaezi, M., Azari, A., Khosravirad, S. R., Shirvanimoghaddam, M., Azari, M. M., Chasaki, D., & Popovski, P. (2022) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2022.3151028 · Cited by 631
Sharma, S. K., & Wang, X. (2020) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2019.2916177 · Cited by 427
“At each instance of transmission, the node selects a slot with the highest Q value and in case of two or more maximum values, a random selection approach can be applied [188].”
BackgroundMethodSelf-citationHussain, F., Hassan, S. A., Hussain, R., & Hossain, E. (2020) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2020.2964534 · Cited by 395
“Collaborative distributed Q-learning was proposed in [94] to reduce congestion in the access channel for resourceconstrained MTC devices in IoT networks.”
Background
Metadata
- Authors
- Sharma, S. K., & Wang, X.
- Affiliations
- Western University (CA)
- Venue
- IEEE Communications Letters
- Published
- 2019-04
- Volume
- 23
- Issue
- 4
- Pages
- 600-603
- ISSN
- 1089-7798 (ISSN-L)
- 1558-2558
- 2373-7891
- OpenAlex
- W2911232490
- Cited by
- 88
- References
- 16
- Citations by year
- 2026: 4 · 2025: 4 · 2024: 19 · 2023: 9 · 2022: 11 · 2021: 22 · 2020: 12 · 2019: 7
- FWCI
- 4.67
- Type
- journal article
- Language
- en
- Open access
- green
- orbilu.uni.lu
- Licence
- Locations
- IEEE Communications Letters (publishedVersion)
- Open Repository and Bibliography (University of Luxembourg) (submittedVersion, cc-by-nc-sa)
- Topics
- Physical Sciences › Engineering › Electrical and Electronic Engineering › IoT Networks and Protocols
- Physical Sciences › Computer Science › Computer Networks and Communications › Wireless Networks and Protocols
- Physical Sciences › Computer Science › Computer Networks and Communications › IoT and Edge/Fog Computing
- Keywords
- machine-type communication, contention-based random access, slotted ALOHA, collision avoidance, resource-constrained devices
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.