Distributed Q-Learning Aided Uplink Grant-Free NOMA for Massive Machine-Type Communications
The DOI matches its Crossref record; 26 of the 26 references it cites were found in a registry or an index.
Abstract
The explosive growth of machine-type communications (MTC) devices poses critical challenges to the existing cellular networks. Therefore, how to support massive MTC devices with limited resources is an urgent problem to be solved. Bursty traffic is an important characteristic of MTC devices, which makes it difficult for agents to learn useful experience and has a negative impact on model convergence. However, most existing reinforcement learning-based literatures assume that devices have saturate data. Towards this end, we propose two distributed Q-learning aided uplink grant-free non-orthogonal multiple access (NOMA) schemes (including all-devices distributed Q-learning (ADDQ) scheme and portion-devices distributed Q-learning (PDDQ) scheme) to maximize the number of accessible devices, where the bursty traffic of massive MTC devices is carefully considered. In order to reduce the dimension of scheduling space and mitigate the impact of bursty traffic, the idea of grouping devices as well as transmission resources and the intermittent learning mode are adopted in our schemes. Extensive numerical results demonstrate the advantages of proposed schemes from multiple perspectives.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Edge AI can also help design AI-native communication strategies for the physical layer (e.g., task-oriented semantic communication [58]) and medium access control layer (e.g., random access protocol [59]).”
From Edge artificial intelligence for 6g: vision, enabling technologies, and applications (2022) · cites this paper as [59]
Citations
Cites
Applications of Deep Reinforcement Learning in Communications and Networking: A Survey
Luong, N. C., Hoang, D. T., Gong, S., Niyato, D., Wang, P., Liang, Y.-C., & Kim, D. I. (2019) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2019.2916583 · Cited by 2,060
Deep Reinforcement Learning Based Resource Allocation for V2V Communications
Ye, H., Li, G. Y., & Juang, B.-H. F. (2019) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2019.2897134 · Cited by 870
Spectrum Sharing in Vehicular Networks Based on Multi-Agent Reinforcement Learning
Liang, L., Ye, H., & Li, G. Y. (2019) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2019.2933962 · Cited by 559 · cited 1×
“This fits well with the nature of RL [23].”
BackgroundDeep Multi-User Reinforcement Learning for Distributed Dynamic Spectrum Access
Naparstek, O., & Cohen, K. (2019) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2018.2879433 · Cited by 477 · cited 3×
“Therefore, the traditional RL-assisted schemes [14]–[17], in which the agent learns in each time slot, are difficult to converge in the mMTC scenario considering the devices’ bursty traffic.”
BackgroundMethodGrant-Free Non-Orthogonal Multiple Access for IoT: A Survey
Shahab, M. B., Abbas, R., Shirvanimoghaddam, M., & Johnson, S. J. (2020) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2020.2996032 · Cited by 453 · cited 2×
“One is that the BS pre-allocates radio resources for each device, and the other is that the device chooses transmission resources by itself [1].”
BackgroundSharma, S. K., & Wang, X. (2020) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2019.2916177 · Cited by 427 · cited 1×
“Without loss of generality, it is assumed that the active probability of each device follows uniform distribution [19] and that P a = { p a 1 , p a 2 , · · · , p aM } is the set of all devices’ active probability.”
BackgroundGhavimi, F., & Chen, H.-H. (2015) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2014.2361626 · Cited by 348 · cited 1×
“In the current LTE/LTE-A system, an active device first requests transmission resources from base station (BS) in a grant-based manner for subsequent data transmission [3], [4].”
BackgroundUplink contention based SCMA for 5G radio access
Au, K., Zhang, L., Nikopour, H., Yi, E., Bayesteh, A., Vilaipornsawai, U., Ma, J., & Zhu, P. (2014) · 2014 IEEE Globecom Workshops (GC Wkshps) · doi:10.1109/glocomw.2014.7063547 · Cited by 289 · cited 2×
“Similar to the existing literatures [9]–[11], [14], [20], we ignore the possibility that some signals in collision can be decoded successfully due to capture effects [9], [22].”
ResultNOMA-Based Random Access With Multichannel ALOHA
Choi, J. (2017) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2017.2766778 · Cited by 276 · cited 3×
“Similar to the existing literatures [9]–[11], [14], [20], we ignore the possibility that some signals in collision can be decoded successfully due to capture effects [9], [22].”
MethodResultBlind detection of SCMA for uplink grant-free multiple-access
Bayesteh, A., Yi, E., Nikopour, H., & Baligh, H. (2014) · 2014 11th International Symposium on Wireless Communications Systems (ISWCS) · doi:10.1109/iswcs.2014.6933472 · Cited by 171 · cited 1×
“Although it is unrealistic for a BS to estimate the channel condition for massive devices in the mMTC scenario, some blind detection and compressive sensing techniques can help BS estimate channel condition by narrowing the range of devices [21].”
Research works printed but not linked: 1
- azar, 2011, Speedy Q-learning, Adv Neural Inf Process Syst, 24, 2411
Cited by
Edge artificial intelligence for 6g: vision, enabling technologies, and applications
Letaief, K. B., Shi, Y., Lu, J., & Lu, J. (2022) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2021.3126076 · Cited by 829
“Edge AI can also help design AI-native communication strategies for the physical layer (e.g., task-oriented semantic communication [58]) and medium access control layer (e.g., random access protocol [59]).”
BackgroundJoint Activity and Channel Estimation for Extra-Large MIMO Systems
Iimori, H., Takahashi, T., Ishibashi, K., de Abreu, G. T. F., David González, G., & Gonsa, O. (2022) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2022.3157271 · Cited by 63
Optimization of Grant-Free NOMA With Multiple Configured-Grants for mURLLC
Yan, L., Deng, Y., Elkashlan, M., Nallanathan, A., & Karagiannidis, G. K. (2022) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2022.3143264 · Cited by 57
Optimal Beamformer Design for Millimeter Wave Dual-Functional Radar-Communication Based V2X Systems
Liu, B., Liu, J., & Kato, N. (2022) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2022.3196089 · Cited by 52
Self-citationNon-Orthogonal Multiple Access Assisted Secure Computation Offloading via Cooperative Jamming
Wu, Y., Ji, G., Wang, T., Qian, L., Lin, B., & Shen, X. (2022) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2022.3167861 · Cited by 51
Metadata
- Authors
- Liu, J., Shi, Z., Zhang, S., & Kato, N.
- Affiliations
- Northwestern Polytechnical University (CN)
- Xidian University (CN)
- Tohoku University (JP)
- Venue
- IEEE Journal on Selected Areas in Communications
- Published
- 2021-07
- Volume
- 39
- Issue
- 7
- Pages
- 2029-2041
- ISSN
- 0733-8716 (ISSN-L)
- 1558-0008
- OpenAlex
- W3160612200
- Cited by
- 53
- References
- 25
- Citations by year
- 2026: 5 · 2025: 3 · 2024: 8 · 2023: 14 · 2022: 18 · 2021: 5
- FWCI
- 3.31
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- IEEE Journal on Selected Areas in Communications (publishedVersion)
- Funders
- National Natural Science Foundation of China (61771374, 61771373, 61801360, 62001393)
- Natural Science Basic Research Program of Shaanxi (2020JC-15, 2020JM-109)
- Fundamental Research Funds for the Central Universities (31020200QD010)
- Xi’an Unmanned System Security and Intelligent Communications ISTC Center
- Special Funds for Central Universities Construction of World-Class Universities (Disciplines) and Special Development Guidance (0639021GH0201024)
- Topics
- Physical Sciences › Engineering › Electrical and Electronic Engineering › IoT Networks and Protocols
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Advanced Wireless Communication Technologies
- Physical Sciences › Computer Science › Computer Networks and Communications › IoT and Edge/Fog Computing
- Keywords
- distributed Q-learning, grant-free access, non-orthogonal multiple access, uplink transmission, massive machine-type communications, bursty traffic, device grouping, resource scheduling
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Distributed Q-Learning Aided Uplink Grant-Free NOMA for Massive Machine-Type Communications. https://app.cochat.ai/papers/10.1109/jsac.2021.3078496
- The paper
- Liu, J., Shi, Z., Zhang, S., & Kato, N. (2021). Distributed Q-Learning Aided Uplink Grant-Free NOMA for Massive Machine-Type Communications. IEEE Journal on Selected Areas in Communications. https://doi.org/10.1109/jsac.2021.3078496
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.