The DOI matches its Crossref record; 58 of the 58 references it cites were found in a registry or an index.
AMP-Inspired Deep Networks for Sparse Linear Inverse Problems
Open accessAbstract
Deep learning has gained great popularity due to its widespread success on many inference problems. We consider the application of deep learning to the sparse linear inverse problem, where one seeks to recover a sparse signal from a few noisy linear measurements. In this paper, we propose two novel neural-network architectures that decouple prediction errors across layers in the same way that the approximate message passing (AMP) algorithms decouple them across iterations: through Onsager correction. First, we propose a “learned AMP” network that significantly improves upon Gregor and LeCun's “learned ISTA.” Second, inspired by the recently proposed “vector AMP” (VAMP) algorithm, we propose a “learned VAMP” network that offers increased robustness to deviations in the measurement matrix from i.i.d. Gaussian. In both cases, we jointly learn the linear transforms and scalar nonlinearities of the network. Interestingly, with i.i.d. signals, the linear transforms and scalar nonlinearities prescribed by the VAMP algorithm coincide with the values learned through back-propagation, leading to an intuitive interpretation of learned VAMP. Finally, we apply our methods to two problems from 5G wireless communications: compressive random access and massive-MIMO channel estimation.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Regression shrinkage and selection via the lasso
Tibshirani, R. (1996) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1996.tb02080.x · Cited by 51,482
A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems
Beck, A., & Teboulle, M. (2009) · SIAM Journal on Imaging Sciences · doi:10.1137/080716542 · Cited by 11,952
Image Super-Resolution Using Deep Convolutional Networks
Dong, C., Loy, C. C., He, K., & Tang, X. (2016) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2439281 · Cited by 10,105
Andrews, J. G., Buzzi, S., Choi, W., Hanly, S. V., Lozano, A., Soong, A. C. K., & Zhang, J. C. (2014) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2014.2328098 · Cited by 8,106
Stable signal recovery from incomplete and inaccurate measurements
Candès, E. J., Romberg, J. K., & Tao, T. (2006) · Communications on Pure and Applied Mathematics · doi:10.1002/cpa.20124 · Cited by 7,036
Atomic Decomposition by Basis Pursuit
Chen, S. S., Donoho, D. L., & Saunders, M. A. (1998) · SIAM Journal on Scientific Computing · doi:10.1137/s1064827596304010 · Cited by 6,845
An iterative thresholding algorithm for linear inverse problems with a sparsity constraint.
Daubechies, I., Defrise, M., & De Mol, C. (2004) · Communications on Pure and Applied Mathematics · doi:10.1002/cpa.20042 · Cited by 4,958
Scaling Up MIMO: Opportunities and Challenges with Very Large Arrays
Rusek, F., Persson, D., Lau, B. K., Larsson, E. G., Marzetta, T. L., & Tufvesson, F. (2013) · IEEE Signal Processing Magazine · doi:10.1109/msp.2011.2178495 · Cited by 4,654
Scaling up MIMO: Opportunities and challenges with very large arrays
Rusek, F., Persson, D., Lau, B. K., Larsson, E. G., Marzetta, T. L., Edfors, O., Tufvesson, F., Print, P., Rusek, F., Persson, D., et al. (2013) · Cited by 4,089
Sparse coding with an overcomplete basis set: A strategy employed by V1?
Olshausen, B. A., & Field, D. J. (1997) · Vision Research · doi:10.1016/s0042-6989(97)00169-7 · Cited by 3,642
Message-passing algorithms for compressed sensing
Donoho, D. L., Maleki, A., & Montanari, A. (2009) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0909892106 · Cited by 2,416
Learning Fast Approximations of Sparse Coding
Gregor, K., & LeCun, Y. (2010) · International Conference on Machine Learning · Cited by 2,400
Random Matrix Theory and Wireless Communications
Tulino, A. M., & Verdú, S. (2004) · Foundations and Trends™ in Communications and Information Theory · doi:10.1561/0100000001 · Cited by 2,034
Compressed sensing : theory and applications
Eldar, Y. C., & Gitta Kutyniok, B. (2012) · Cited by 2,027
Compressed Sensing: Theory and Applications
(2012) · Cambridge University Press eBooks · doi:10.1017/cbo9780511794308 · Cited by 1,660
Splines: a perfect fit for signal and image processing
Unser, M. (1999) · IEEE Signal Processing Magazine · doi:10.1109/79.799930 · Cited by 1,633
Image denoising: Can plain neural networks compete with BM3D?
Burger, H. C., Schuler, C. J., & Harmeling, S. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6247952 · Cited by 1,325
Srivastava, R. K., Greff, K., & Schmidhuber, J. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1507.06228 · Cited by 1,246
Wainwright, M. J. (2009) · IEEE Transactions on Information Theory · doi:10.1109/tit.2009.2016018 · Cited by 1,198
Deconstructing multiantenna fading channels
Sayeed, A. M. (2002) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2002.803324 · Cited by 1,120
Generalized approximate message passing for estimation with random linear mixing
Rangan, S. (2011) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.2011.6033942 · Cited by 1,103
Self-citationResidual networks behave like ensembles of relatively shallow networks
Veit, A., Wilber, M. J., & Belongie, S. (2016) · neural information processing systems · Cited by 851
Deep Networks for Image Super-Resolution with Sparse Prior
Wang, Z., Liu, D., Yang, J., Han, W., & Huang, T. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.50 · Cited by 784
The Dynamics of Message Passing on Dense Graphs, with Applications to Compressed Sensing
Bayati, M., & Montanari, A. (2011) · IEEE Transactions on Information Theory · doi:10.1109/tit.2010.2094817 · Cited by 780
ReconNet: Non-Iterative Reconstruction of Images from Compressively Sensed Measurements
Kulkarni, K., Lohit, S., Turaga, P., Kerviche, R., & Ashok, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.55 · Cited by 768
Chambolle, A., De Vore, R. A., Lee, N.-Y., & Lucier, B. J. (1998) · IEEE Transactions on Image Processing · doi:10.1109/83.661182 · Cited by 751
Residual Networks Behave Like Ensembles of Relatively Shallow Networks
Veit, A., Wilber, M., & Belongie, S. (2016) · arXiv · doi:10.48550/arxiv.1605.06431 · Cited by 705
Compressed Sensing: List of contributors
Eldar, Y. C., & Kutyniok, G. (2012) · Cited by 600
Shrinkage Fields for Effective Image Restoration
Schmidt, U., & Roth, S. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.349 · Cited by 543
Message passing algorithms for compressed sensing: I. motivation and construction
Donoho, D. L., Maleki, A., & Montanari, A. (2010) · IEEE Information Theory Workshop 2010 (ITW 2010) · doi:10.1109/itwksps.2010.5503193 · Cited by 542
Expectation-Maximization Gaussian-Mixture Approximate Message Passing
Vila, J. P., & Schniter, P. (2013) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2013.2272287 · Cited by 501
Self-citationDeep Unfolding: Model-Based Inspiration of Novel Deep Architectures
Hershey, J. R., Roux, J. L., & Weninger, F. (2014) · arXiv · doi:10.48550/arxiv.1409.2574 · Cited by 412 · cited 1×
“For example, convolutional neural networks and stacked denoising autoencoders have been applied to speech enhancement [27], image denoising [28], image deblurring [29], [30], image super resolution [31], 3D imaging [32], compressive imaging [33]– [35], and video compressive sensing [36].”
MethodA deep learning approach to structured signal recovery
Mousavi, A., Patel, A. B., & Baraniuk, R. G. (2015) · 2015 53rd Annual Allerton Conference on Communication, Control, and Computing (Allerton) · doi:10.1109/allerton.2015.7447163 · Cited by 355
A Machine Learning Approach for Non-blind Image Deconvolution
Schuler, C. J., Burger, H. C., Harmeling, S., & Scholkopf, B. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.142 · Cited by 319
Channel Estimation for Massive MIMO Using Gaussian-Mixture Bayesian Learning
Wen, C.-K., Jin, S., Wong, K.-K., Chen, J.-C., & Ting, P. (2015) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2014.2365813 · Cited by 310
Compressive Phase Retrieval via Generalized Approximate Message Passing
Schniter, P., & Rangan, S. (2015) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2014.2386294 · Cited by 279
Self-citationLearning to invert: Signal recovery via Deep Convolutional Networks
Mousavi, A., & Baraniuk, R. G. (2017) · 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2017.7952561 · Cited by 274
Deep fully-connected networks for video compressive sensing
Iliadis, M., Spinoulas, L., & Katsaggelos, A. K. (2018) · Digital Signal Processing · doi:10.1016/j.dsp.2017.09.010 · Cited by 210
Universality in polytope phase transitions and message passing algorithms
Bayati, M., Lelarge, M., & Andrea Montanari, A. (2015) · The Annals of Applied Probability · doi:10.1214/14-aap1010 · Cited by 209
Compressive sensing based multi‐user detection for machine‐to‐machine communication
Bockelmann, C., Schepker, H., & Dekorsy, A. (2013) · Transactions on Emerging Telecommunications Technologies · doi:10.1002/ett.2633 · Cited by 163
Expectation-maximization Bernoulli-Gaussian approximate message passing
Vila, J. P., & Schniter, P. (2011) · Asilomar Conference on Signals, Systems and Computers (ACSSC) · doi:10.1109/acssc.2011.6190117 · Cited by 153
Self-citationVector approximate message passing for the generalized linear model
Schniter, P., Rangan, S., & Fletcher, A. K. (2016) · Asilomar Conference on Signals, Systems and Computers (ACSSC) · doi:10.1109/acssc.2016.7869633 · Cited by 147
Self-citationSparse Signal Processing Concepts for Efficient 5G System Design
Wunder, G., Boche, H., Strohmer, T., & Jung, P. (2015) · IEEE Access · doi:10.1109/access.2015.2407194 · Cited by 146
Maximal Sparsity with Deep Networks?Preprint
Xin, B., Wang, Y., Gao, W., & Wipf, D. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.01636 · Cited by 139
Message-Passing De-Quantization With Applications to Compressed Sensing
Kamilov, U. S., Goyal, V. K., & Rangan, S. (2012) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2012.2217334 · Cited by 134
Self-citationLearning Optimal Nonlinearities for Iterative Thresholding Algorithms
Kamilov, U. S., & Mansour, H. (2016) · IEEE Signal Processing Letters · doi:10.1109/lsp.2016.2548245 · Cited by 105
Onsager-corrected deep learning for sparse linear inverse problems
Borgerding, M., & Schniter, P. (2016) · IEEE Global Conference on Signal and Information Processing · doi:10.1109/globalsip.2016.7905837 · Cited by 95
Self-citationThe dynamics of message passing on dense graphs, with applications to compressed sensing
Bayati, M., & Andrea Montanari, A. (2010) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.2010.5513529 · Cited by 92
Hybrid Approximate Message Passing
Rangan, S., Fletcher, A. K., Goyal, V. K., Byrne, E., & Schniter, P. (2017) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2017.2713759 · Cited by 87
Self-citationNear Optimal Compressed Sensing Without Priors: Parametric SURE Approximate Message Passing
Guo, C., & Davies, M. E. (2015) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2015.2408569 · Cited by 68
Expectation-maximization Gaussian-mixture approximate message passing
Vila, J. P., & Schniter, P. (2012) · Conference on Information Sciences and Systems · doi:10.1109/ciss.2012.6310932 · Cited by 61
Self-citationMulti-layer generalized linear estimation
Manoel, A., Krząkała, F., Mézard, M., & Zdeborová, L. (2017) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.2017.8006899 · Cited by 46
ReconNet: Non-Iterative Reconstruction of Images from Compressively Sensed Random MeasurementsPreprint
Kulkarni, K., Lohit, S., Turaga, P. K., Kerviche, R., & Ashok, A. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1601.06892 · Cited by 39
Multi-Layer Generalized Linear EstimationPreprint
Manoel, A., Krząkała, F., Mézard, M., & Zdeborová, L. (2017) · HAL · Cited by 37
Learning and free energies for vector approximate message passing
Fletcher, A. K., & Schniter, P. (2017) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2017.7952957 · Cited by 33
Self-citationLearning Efficient Structured Sparse ModelsPreprint
Bronstein, A. M., Sprechmann, P., & Sapiro, G. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1206.4649 · Cited by 28
Cited by
Deep Learning in Mobile and Wireless Networking: A Survey
Zhang, C., Patras, P., & Haddadi, H. (2019) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2019.2904897 · Cited by 1,768
ISTA-Net: Interpretable Optimization-Inspired Deep Network for Image Compressive Sensing
Zhang, J., & Ghanem, B. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00196 · Cited by 1,309
Deep Learning Techniques for Inverse Problems in Imaging
Ongie, G., Jalal, A., Metzler, C. A., Baraniuk, R. G., Dimakis, A. G., & Willett, R. (2020) · IEEE Journal on Selected Areas in Information Theory · doi:10.1109/jsait.2020.2991563 · Cited by 577
Model-Driven Deep Learning for MIMO Detection
He, H., Wen, C.-K., Jin, S., & Li, G. Y. (2020) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2020.2976585 · Cited by 454
Deep Learning for Physical-Layer 5G Wireless Techniques: Opportunities, Challenges and Solutions
Huang, H., Guo, S., Gui, G., Yang, Z., Zhang, J., Sari, H., & Adachi, F. (2020) · IEEE Wireless Communications · doi:10.1109/mwc.2019.1900027 · Cited by 431
Metadata
- Authors
- Borgerding, M., Schniter, P., & Rangan, S.
- Venue
- IEEE Transactions on Signal Processing
- Published
- 2017-08-15
- Volume
- 65
- Issue
- 16
- Pages
- 4293-4308
- ISSN
- 1053-587X (ISSN-L)
- 1941-0476
- OpenAlex
- W2619204584
- Cited by
- 455
- References
- 58
- FWCI
- 29.99
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.