Joint Deep Reinforcement Learning and Unfolding: Beam Selection and Precoding for mmWave Multiuser MIMO With Lens Arrays
Open accessThe DOI matches its Crossref record; 40 of the 41 references it cites were found in a registry or an index.
Abstract
The millimeter wave (mmWave) multiuser multiple-input multiple-output (MU-MIMO) systems with discrete lens arrays (DLA) have received great attention due to their simple hardware implementation and excellent performance. In this work, we investigate the joint design of beam selection and digital precoding matrices for mmWave MU-MIMO systems with DLA to maximize the sum-rate subject to the transmit power constraint and the constraints of the selection matrix structure. The investigated non-convex problem with discrete variables and coupled constraints is challenging to solve and an efficient framework of joint neural network (NN) design is proposed to tackle it. Specifically, the proposed framework consists of a deep reinforcement learning (DRL)-based NN and a deep-unfolding NN, which are employed to optimize the beam selection and digital precoding matrices, respectively. As for the DRL-based NN, we formulate the beam selection problem as a Markov decision process and a double deep Q-network algorithm is developed to solve it. The base station is considered to be an agent, where the state, action, and reward function are carefully designed. Regarding the design of the digital precoding matrix, we develop an iterative weighted minimum mean-square error algorithm induced deep-unfolding NN, which unfolds this algorithm into a layer-wise structure with introduced trainable parameters. Simulation results verify that this jointly trained NN remarkably outperforms the existing itera…
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“To reduce the model and sample complexity, as well as improve the performance and interpretability, unfolded neural networks were developed in [267]- [269] to parameterize the iterative policy via unfolding one iteration of the existing structured algorithm into one layer of a neural network.”
From Edge artificial intelligence for 6g: vision, enabling technologies, and applications (2022) · cites this paper as [268]
Citations
Cites
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357 · cited 1×
“In addition, deep reinforcement learning (DRL) [27] has also been widely applied to solve the high-dimensional non-convex optimization problems.”
MethodNoncooperative Cellular Wireless with Unlimited Numbers of Base Station Antennas
Marzetta, T. L. (2010) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2010.092810.091092 · Cited by 6,302 · cited 1×
“3087233 of antennas at the transceiver in a cellular network, where the massive multiple-input multiple-output (MIMO) techniques can be readily realized to perform highly directional transmissions [2]–[4].”
BackgroundScaling Up MIMO: Opportunities and Challenges with Very Large Arrays
Rusek, F., Persson, D., Lau, B. K., Larsson, E. G., Marzetta, T. L., & Tufvesson, F. (2013) · IEEE Signal Processing Magazine · doi:10.1109/msp.2011.2178495 · Cited by 4,654 · cited 1×
“3087233 of antennas at the transceiver in a cellular network, where the massive multiple-input multiple-output (MIMO) techniques can be readily realized to perform highly directional transmissions [2]–[4].”
BackgroundScaling up MIMO: Opportunities and challenges with very large arrays
Rusek, F., Persson, D., Lau, B. K., Larsson, E. G., Marzetta, T. L., Edfors, O., Tufvesson, F., Print, P., Rusek, F., Persson, D., et al. (2013) · Cited by 4,089
Five disruptive technology directions for 5G
Boccardi, F., Heath, R. W., Lozano, A., Marzetta, T. L., & Popovski, P. (2014) · IEEE Communications Magazine · doi:10.1109/mcom.2014.6736746 · Cited by 3,709
Spatially Sparse Precoding in Millimeter Wave MIMO Systems
Ayach, O. E., Rajagopal, S., Abu-Surra, S., Pi, Z., & Heath, R. W. (2014) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2014.011714.130846 · Cited by 3,557 · cited 1×
“The hybrid analog-digital (AD) transceiver consisting of cascaded baseband digital precoder and RF analog beamformer has been proposed as a trade-off between cost and performance, which employs less RF chains than the number of antennas, while imposing a constant modulus constraint on the analog beamforming matrix [5].”
BackgroundDeep Reinforcement Learning with Double Q-Learning
Van Hasselt, H., Guez, A., & Silver, D. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.10295 · Cited by 3,318 · cited 2×
“We apply the DDQN [36] to learn the target value y t to avoid the overestimation and make the NN stable, which is expressed as where θ (cid:48) and θ denote the parameters of the two NNs, which have been applied to select and evaluate the action, respectively.”
MethodShi, Q., Razaviyayn, M., Luo, Z.-Q., & He, C. (2011) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2011.2147784 · Cited by 2,129 · cited 5×
“Given the beam selection matrix, the problem of the digital precoding design is a classic sum-rate maximization problem with power constraints that can be settled by the iterative WMMSE algorithm [36].”
BackgroundMethodInfluentialHuang, L., Bi, S., & Zhang, Y.-J. A. (2020) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2019.2928811 · Cited by 1,040 · cited 5×
“The DQN has been widely applied in mobile offloading [28], dynamic channel access [29], radio access [30], resource allocation [31], [32], and mobile-edge computing (MEC) [33].”
BackgroundMethodInfluentialLearning to Optimize: Training Deep Neural Networks for Interference Management
Sun, H., Chen, X., Shi, Q., Hong, M., Fu, X., & Sidiropoulos, N. D. (2018) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2018.2866382 · Cited by 874 · cited 2×
“Recently, the machine learning techniques emerge as an efficient method and there are a number of studies employing the neural networks (NNs) for solving the investigated problems [16], such as precoding design [17], beam selection [18]–[20], and resource allocation [21].”
BackgroundMethod
Cited by
Edge artificial intelligence for 6g: vision, enabling technologies, and applications
Letaief, K. B., Shi, Y., Lu, J., & Lu, J. (2022) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2021.3126076 · Cited by 829
“To reduce the model and sample complexity, as well as improve the performance and interpretability, unfolded neural networks were developed in [267], [268], [269] to parameterize the iterative policy via unfolding one iteration of the existing structured algorithm into one layer of a neural network.”
MethodA Survey of Beam Management for mmWave and THz Communications Towards 6G
Xue, Q., Ji, C., Ma, S., Guo, J., Xu, Y., Chen, Q., & Zhang, W. (2024) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2024.3361991 · Cited by 280
“In [88], the joint design of beam selection and digital precoding for mmWave MU-MIMO systems was investigated to maximize the sum-rate.”
MethodLearning Decentralized Wireless Resource Allocations With Graph Neural Networks
Wang, Z., Eisen, M., & Ribeiro, A. (2022) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2022.3163626 · Cited by 100
“And, if the parametrization is suitably chosen, they can scale to networks with large numbers of transceivers [17]–[27].”
Sun, R., Cheng, N., Li, C., Quan, W., Zhou, H., Wang, Y., Zhang, W., & Shen, X. (2025) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2025.3574765 · Cited by 89
“To reduce online processing time and keep interpretability, a general framework of iterative WMMSE-unfolded customized DNN is developed in matrix form for the multi-user MIMO scenario [104], [105], [116].”
Deep Unfolding Hybrid Beamforming Designs for THz Massive MIMO Systems
Nguyen, N. T., Ma, M., Lavi, O., Shlezinger, N., Eldar, Y. C., Swindlehurst, A. L., & Juntti, M. J. (2023) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2023.3322852 · Cited by 79
“The former relies mainly on the learning capability of deep neural networks (DNNs) [24], [25], [26], convolutional neural networks (CNNs) [27], [28], [29], [30], [37], [38], [39], [40], or deep reinforcement learning [41], [42] to generate HBF beamformers.”
Background
Metadata
- Authors
- Hu, Q., Liu, Y., Cai, Y., Yu, G., & Ding, Z.
- Affiliations
- Zhejiang University (CN)
- University of California, Davis (US)
- Venue
- IEEE Journal on Selected Areas in Communications
- Published
- 2021-08
- Volume
- 39
- Issue
- 8
- Pages
- 2289-2304
- ISSN
- 0733-8716 (ISSN-L)
- 1558-0008
- OpenAlex
- W3167193327
- Cited by
- 64
- References
- 41
- Citations by year
- 2026: 7 · 2025: 12 · 2024: 11 · 2023: 18 · 2022: 11 · 2021: 5
- FWCI
- 3.31
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE Journal on Selected Areas in Communications (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Natural Science Foundation of China (61971376, 61831004)
- Zhejiang Provincial Natural Science Foundation for Distinguished Young Scholars (LR19F010002)
- Topics
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Millimeter-Wave Propagation and Modeling
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Advanced Wireless Communication Technologies
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Advanced MIMO Systems Optimization
- Keywords
- millimeter wave, multi-user MIMO, beam selection, digital precoding, deep reinforcement learning, deep unfolding, double deep Q-network, sum rate maximization, Markov decision process, neural network design
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Joint Deep Reinforcement Learning and Unfolding: Beam Selection and Precoding for mmWave Multiuser MIMO With Lens Arrays. https://app.cochat.ai/papers/10.1109/jsac.2021.3087233
- The paper
- Hu, Q., Liu, Y., Cai, Y., Yu, G., & Ding, Z. (2021). Joint Deep Reinforcement Learning and Unfolding: Beam Selection and Precoding for mmWave Multiuser MIMO With Lens Arrays. IEEE Journal on Selected Areas in Communications. https://doi.org/10.1109/jsac.2021.3087233
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.