The DOI matches its Crossref record; 91 of the 91 references it cites were found in a registry or an index.
A Unified Algorithmic Framework for Block-Structured Optimization Involving Big Data: With applications in machine learning and signal processing
Abstract
This article presents a powerful algorithmic framework for big data optimization, called the block successive upper-bound minimization (BSUM). The BSUM includes as special cases many well-known methods for analyzing massive data sets, such as the block coordinate descent (BCD) method, the convex-concave procedure (CCCP) method, the block coordinate proximal gradient (BCPG) method, the nonnegative matrix factorization (NMF) method, the expectation maximization (EM) method, etc. In this article, various features and properties of the BSUM are discussed from the viewpoint of design flexibility, computational efficiency, parallel/distributed implementation, and the required communication overhead. Illustrative examples from networking, signal processing, and machine learning are presented to demonstrate the practical performance of the BSUM framework.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Fast and accurate short read alignment with Burrows-Wheeler transform
Li, H., & Durbin, R. (2009) · Bioinformatics · doi:10.1093/bioinformatics/btp324 · Cited by 58,607
Maximum Likelihood from Incomplete Data Via the EM Algorithm
Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1977.tb01600.x · Cited by 50,444
Donoho, D. L. (2006) · IEEE Transactions on Information Theory · doi:10.1109/tit.2006.871582 · Cited by 23,994
Ultrafast and memory-efficient alignment of short DNA sequences to the human genome.
Langmead, B., Trapnell, C., Pop, M., & Salzberg, S. L. (2009) · Genome Biology · doi:10.1186/gb-2009-10-3-r25 · Cited by 21,849
Regularization Paths for Generalized Linear Models via Coordinate Descent
Friedman, J., Hastie, T., & Tibshirani, R. (2010) · Journal of Statistical Software · doi:10.18637/jss.v033.i01 · Cited by 19,927
Boyd, S., Parikh, N., Chu, E., Peleato, B., & Eckstein, J. (2011) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000016 · Cited by 17,075
Donoho, D. L. (2004) · Cited by 16,703
Candes, E. J., Romberg, J., & Tao, T. (2006) · IEEE Transactions on Information Theory · doi:10.1109/tit.2005.862083 · Cited by 15,639
Boyd, S. (2010) · now publishers, Inc. eBooks · doi:10.1561/9781601984616 · Cited by 12,677
A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems
Beck, A., & Teboulle, M. (2009) · SIAM Journal on Imaging Sciences · doi:10.1137/080716542 · Cited by 11,952
Tensor Decompositions and Applications
Kolda, T. G., & Bader, B. W. (2009) · SIAM Review · doi:10.1137/07070111x · Cited by 10,301
ALGORITHMS FOR NON-NEGATIVE MATRIX FACTORIZATION
Lee, D. D., & Seung, H. S. (2000) · neural information processing systems · Cited by 9,878
$rm K$-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation
Aharon, M., Elad, M., & Bruckstein, A. (2006) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2006.881199 · Cited by 9,451
Parallel and Distributed Computation: Numerical Methods
Bertsekas, D. P., & Tsitsiklis, J. N. (1989) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 5,293
Carroll, J. D., & Chang, J.-J. (1970) · Psychometrika · doi:10.1007/bf02310791 · Cited by 4,678
Figueiredo, M. A. T., Nowak, R. D., & Wright, S. J. (2007) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2007.910281 · Cited by 3,524
Discriminant Analysis and Statistical Pattern Recognition.
Hawkins, D. M., & McLachlan, G. J. (1993) · Journal of the American Statistical Association · doi:10.2307/2290355 · Cited by 2,633
Efficient sparse coding algorithms
Lee, H., Battle, A., Raina, R., & Ng, A. Y. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0105 · Cited by 2,394
Online Learning for Matrix Factorization and Sparse Coding
Mairal, J., Bach, F., Ponce, J., & Sapiro, G. (2009) · arXiv (Cornell University) · doi:10.48550/arxiv.0908.0050 · Cited by 2,258
Harshman, R. A. (1970) · Cited by 2,255
Proximal Splitting Methods in Signal Processing
Combettes, P. L., & Pesquet, J.-C. (2011) · Springer Optimization and Its Applications · doi:10.1007/978-1-4419-9569-8_10 · Cited by 2,183
Shi, Q., Razaviyayn, M., Luo, Z.-Q., & He, C. (2011) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2011.2147784 · Cited by 2,129 · cited 5×
“It also includes as special cases many well-known signal processing algorithms such as the family of interference pricing algorithms [33], [34] and the weighted minimum mean square error algorithms [12], [35] for interference management in large-scale wireless systems.”
BackgroundMethodInfluentialSelf-citationOnline dictionary learning for sparse coding
Mairal, J., Bach, F., Ponce, J., & Sapiro, G. (2009) · Proceedings of the 26th Annual International Conference on Machine Learning · doi:10.1145/1553374.1553463 · Cited by 2,114
Convergence of a Block Coordinate Descent Method for Nondifferentiable Minimization
Tseng, P. (2001) · Journal of Optimization Theory and Applications · doi:10.1023/a:1017501703105 · Cited by 1,989
Hunter, D. R., & Lange, K. (2004) · The American Statistician · doi:10.1198/0003130042836 · Cited by 1,783
Numerical Methods for Nonlinear Variational Problems
Glowinski, R. (1984) · Springer eBooks · doi:10.1007/978-3-662-12613-4 · Cited by 1,728
Wright, S. J. (2015) · Mathematical Programming · doi:10.1007/s10107-015-0892-3 · Cited by 1,438
A Unified Convergence Analysis of Block Successive Minimization Methods for Nonsmooth Optimization
Razaviyayn, M., Hong, M., & Luo, Z.-Q. (2013) · SIAM Journal on Optimization · doi:10.1137/120891009 · Cited by 1,304
Self-citationGradient methods for minimizing composite functions
Nesterov, Y. (2013) · Mathematical Programming · doi:10.1007/s10107-012-0629-5 · Cited by 1,298
Iteratively reweighted least squares minimization for sparse recovery
Daubechies, I., DeVore, R., Fornasier, M., & Güntürk, C. S. (2010) · Communications on Pure and Applied Mathematics · doi:10.1002/cpa.20303 · Cited by 1,295
Efficiency of Coordinate Descent Methods on Huge-Scale Optimization Problems
Nesterov, Y. (2012) · SIAM Journal on Optimization · doi:10.1137/100802001 · Cited by 1,187
Xu, Y., & Yin, W. (2013) · SIAM Journal on Imaging Sciences · doi:10.1137/120887795 · Cited by 1,160
Yuille, A. L., & Rangarajan, A. (2003) · Neural Computation · doi:10.1162/08997660360581958 · Cited by 1,151
Streaming fragment assignment for real-time analysis of sequencing experiments
Roberts, A., & Pachter, L. (2013) · Nature Methods · doi:10.1038/nmeth.2251 · Cited by 1,063
Weighted sum-rate maximization using weighted MMSE for MIMO-BC beamforming design
Christensen, S. S., Agarwal, R., De Carvalho, E., & Cioffi, J. M. (2008) · IEEE Transactions on Wireless Communications · doi:10.1109/t-wc.2008.070851 · Cited by 1,014
Exact matrix completion via convex optimization
Candès, E., & Recht, B. (2012) · Communications of the ACM · doi:10.1145/2184319.2184343 · Cited by 996
Technical Note—A General Inner Approximation Algorithm for Nonconvex Mathematical Programs
Marks, B. R., & Wright, G. P. (1978) · Operations Research · doi:10.1287/opre.26.4.681 · Cited by 985
Discriminant Analysis and Statistical Pattern Recognition
Sapatinas, T. (2005) · Journal of the Royal Statistical Society Series A: Statistics in Society · doi:10.1111/j.1467-985x.2005.00368_10.x · Cited by 942
Dynamic Spectrum Management: Complexity and Duality
Luo, Z.-Q., & Zhang, S. (2008) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2007.914876 · Cited by 929
Self-citationOptimization for Machine Learning
(2011) · The MIT Press eBooks · doi:10.7551/mitpress/8996.001.0001 · Cited by 875
Shi, Q., Razaviyayn, M., Luo, Z.-Q., & He, C. (2011) · 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2011.5946304 · Cited by 873
Self-citationHong, M., Luo, Z.-Q., & Razaviyayn, M. (2016) · SIAM Journal on Optimization · doi:10.1137/140990309 · Cited by 841
Self-citationOn the convergence of the block nonlinear Gauss–Seidel method under convex constraints
Grippo, L., & Sciandrone, M. (2000) · Operations Research Letters · doi:10.1016/s0167-6377(99)00074-7 · Cited by 678
Algorithms for Nonnegative Matrix Factorization with the β-Divergence
Févotte, C., & Idier, J. (2011) · Neural Computation · doi:10.1162/neco_a_00168 · Cited by 663
Richtárik, P., & Takáč, M. (2014) · Mathematical Programming · doi:10.1007/s10107-012-0614-z · Cited by 654
Håstad, J. (1990) · Journal of Algorithms · doi:10.1016/0196-6774(90)90014-6 · Cited by 584
On the Convergence of Block Coordinate Descent Type Methods
Beck, A., & Tetruashvili, L. (2013) · SIAM Journal on Optimization · doi:10.1137/120887679 · Cited by 572
Non-negative tensor factorization with applications to statistics and computer vision
Shashua, A., & Hazan, T. (2005) · Proceedings of the 22nd international conference on Machine learning - ICML '05 · doi:10.1145/1102351.1102451 · Cited by 490
Parallel coordinate descent methods for big data optimization
Richtárik, P., & Takáč, M. (2016) · Mathematical Programming · doi:10.1007/s10107-015-0901-6 · Cited by 396
Penalized Classification using Fisher’s Linear Discriminant
Witten, D. M., & Tibshirani, R. (2011) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.1467-9868.2011.00783.x · Cited by 384
Successive Convex Approximation: Analysis and Applications
Razaviyayn, M. (2014) · University of Minnesota Digital Conservancy (University of Minnesota) · Cited by 356
Self-citationA regularization approach to joint blur identification and image restoration
You, Y.-L., & Kaveh, M. (1996) · IEEE Transactions on Image Processing · doi:10.1109/83.491316 · Cited by 356
Kim, J., He, Y., & Park, H. (2014) · Journal of Global Optimization · doi:10.1007/s10898-013-0035-4 · Cited by 352
Decomposition by Partial Linearization: Parallel Optimization of Multi-Agent Systems
Scutari, G., Facchinei, F., Song, P., Palomar, D. P., & Pang, J.-S. (2014) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2013.2293126 · Cited by 338
Self-citationOn search directions for minimization algorithms
Powell, M. J. D. (1973) · Mathematical Programming · doi:10.1007/bf01584660 · Cited by 338
Optimal Resource Allocation for MIMO Ad Hoc Cognitive Radio Networks
Kim, S.-J., & Giannakis, G. B. (2011) · IEEE Transactions on Information Theory · doi:10.1109/tit.2011.2120270 · Cited by 301
Cevher, V., Becker, S., & Schmidt, M. (2014) · IEEE Signal Processing Magazine · doi:10.1109/msp.2014.2329397 · Cited by 300
Hong, M., Sun, R., Baligh, H., & Luo, Z.-Q. (2013) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2013.130211 · Cited by 271
Self-citationA Guide to Sample Average Approximation
Kim, S., Pasupathy, R., & Henderson, S. G. (2015) · International Series in Operations Research & Management Science · doi:10.1007/978-1-4939-1384-8_8 · Cited by 268
Accelerated, Parallel, and Proximal Coordinate Descent
Fercoq, O., & Richtárik, P. (2015) · SIAM Journal on Optimization · doi:10.1137/130949993 · Cited by 265
Beck, A. (2015) · SIAM Journal on Optimization · doi:10.1137/13094829x · Cited by 221
On the complexity analysis of randomized block-coordinate descent methods
Lu, Z., & Xiao, L. (2015) · Mathematical Programming · doi:10.1007/s10107-014-0800-2 · Cited by 203
An Asynchronous Parallel Stochastic Coordinate Descent AlgorithmPreprint
Liu, J., Wright, S. J., Ré, C., Bittorf, V., & Sridhar, S. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1311.1873 · Cited by 183
Distributed Coordinate Descent Method for Learning with Big Data
Richtárik, P., & Takáč, M. (2013) · Edinburgh Research Explorer (University of Edinburgh) · doi:10.48550/arxiv.1310.2059 · Cited by 182
Iteration complexity analysis of block coordinate descent methods
Hong, M., Wang, X., Razaviyayn, M., & Luo, Z.-Q. (2017) · Mathematical Programming · doi:10.1007/s10107-016-1057-8 · Cited by 163
Self-citationParallel Selective Algorithms for Nonconvex Big Data Optimization
Facchinei, F., Scutari, G., & Sagratella, S. (2015) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2015.2399858 · Cited by 162
Hong, M., Luo, Z.-Q. T., & Razaviyayn, M. (2015) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2015.7178689 · Cited by 159
Self-citationOptimization with First-Order Surrogate FunctionsPreprint
Mairal, J. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1305.3120 · Cited by 138
Lin, Q., Lu, Z., & Xiao, L. (2015) · SIAM Journal on Optimization · doi:10.1137/141000270 · Cited by 135
On the Nonasymptotic Convergence of Cyclic Coordinate Descent Methods
Saha, A., & Tewari, A. (2013) · SIAM Journal on Optimization · doi:10.1137/110840054 · Cited by 134
Pillow, J. W., Ahmadian, Y., & Paninski, L. (2010) · Neural Computation · doi:10.1162/neco_a_00058 · Cited by 133
Razaviyayn, M., Sanjabi, M., & Luo, Z.-Q. T. (2016) · Mathematical Programming · doi:10.1007/s10107-016-1021-7 · Cited by 118
Self-citationNear-optimal RNA-Seq quantificationPreprint
Bray, N., Pimentel, H., Melsted, P., & Pachter, L. S. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1505.02710 · Cited by 115
Coordinate Descent Converges Faster with the Gauss-Southwell Rule Than Random SelectionPreprint
Nutini, J., Schmidt, M., Laradji, I., Friedlander, M. P., & Koepke, H. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.00552 · Cited by 110
Stochastic Majorization-Minimization Algorithms for Large-Scale OptimizationPreprint
Mairal, J. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1306.4650 · Cited by 105
Guaranteed Matrix Completion via Nonconvex Factorization
Sun, R., & Luo, Z.-Q. T. (2015) · Annual Symposium on Foundations of Computer Science · doi:10.1109/focs.2015.25 · Cited by 102
Self-citationRank Regularization and Bayesian Inference for Tensor Completion and Extrapolation
Bazerque, J. A., Mateos, G., & Giannakis, G. B. (2013) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2013.2278516 · Cited by 100
Parallel Successive Convex Approximation for Nonsmooth Nonconvex OptimizationPreprint
Razaviyayn, M., Hong, M., Luo, Z.-Q. T., & Pang, J. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1406.3665 · Cited by 98
Self-citationHong, M., Chang, T., Wang, X., Razaviyayn, M., Ma, S., & Luo, Z.-Q. T. (2020) · Mathematics of Operations Research · doi:10.1287/moor.2019.1010 · Cited by 94
Self-citationSignal Processing and Optimal Resource Allocation for the Interference Channel
Hong, M., & Luo, Z.-Q. T. (2014) · Academic press library in signal processing · doi:10.1016/b978-0-12-396500-4.00008-9 · Cited by 88
Self-citationParallel and distributed sparse optimization
Peng, Z., Yan, M., & Yin, W. (2013) · Asilomar Conference on Signals, Systems and Computers (ACSSC) · doi:10.1109/acssc.2013.6810364 · Cited by 87
Monotonic convergence of distributed interference pricing in wireless networks
Shi, C., Berry, R., & Honig, M. L. (2009) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.2009.5205801 · Cited by 86
Hong, M., Li, Q., & Liu, Y. (2015) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2015.2489640 · Cited by 54
Self-citationCross-Layer Provision of Future Cellular Networks: A WMMSE-based approach
Baligh, H., Hong, M., Liao, W.-C., Luo, Z.-Q. T., Razaviyayn, M., Sanjabi, M., & Sun, R. (2014) · IEEE Signal Processing Magazine · doi:10.1109/msp.2014.2335237 · Cited by 46
Self-citationA stochastic weighted MMSE approach to sum rate maximization for a MIMO interference channel
Razaviyayn, M., Boroujeni, M. S., & Luo, Z.-Q. T. (2013) · IEEE International Workshop on Signal Processing Advances in Wireless Communications (SPAWC) · doi:10.1109/spawc.2013.6612065 · Cited by 42
Self-citationDistributed interference pricing with MISO channels
Shi, C., Berry, R., & Honig, M. L. (2008) · Allerton Conference on Communication, Control, and Computing · doi:10.1109/allerton.2008.4797605 · Cited by 39
Interactive Sensing and Decision Making in Social Networks
Krishnamurthy, V., Gharehshiran, O. N., & Hamdi, M. (2014) · Foundations and Trends® in Signal Processing · doi:10.1561/2000000048 · Cited by 38
Cui, S. R., Hero, A. O., Luo, Z.-Q. T., & Moura, J. M. F. (2015) · Cambridge University Press eBooks · doi:10.1017/cbo9781316162750 · Cited by 29
Self-citationJoint User Grouping and Transceiver Design in a MIMO Interfering Broadcast Channel
Razaviyayn, M., Baligh, M. H., Callard, A., & Luo, Z.-Q. T. (2013) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2013.2284753 · Cited by 26
Self-citationFlexible parallel algorithms for big data optimization
Facchinei, F., Sagratella, S., & Scutari, G. (2014) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2014.6854999 · Cited by 25
Randomized Block Coordinate Non-Monotone Gradient Method for a Class of Nonlinear Programming
Lu, Z., & Xiao, L. (2013) · Cited by 19
Cited by
Joint Trajectory and Communication Design for Multi-UAV Enabled Wireless Networks
Wu, Q., Zeng, Y., & Zhang, R. (2018) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2017.2789293 · Cited by 1,978
Accessing From the Sky: A Tutorial on UAV Communications for 5G and Beyond
Zeng, Y., Wu, Q., & Zhang, R. (2019) · Proceedings of the IEEE · doi:10.1109/jproc.2019.2952892 · Cited by 1,589
Intelligent Reflecting Surface Meets OFDM: Protocol Design and Rate Maximization
Yang, Y., Zheng, B., Zhang, S., & Zhang, R. (2020) · IEEE Transactions on Communications · doi:10.1109/tcomm.2020.2981458 · Cited by 775
DC programming and DCA: thirty years of developments
Le Thi, H. A., & Pham Dinh, T. (2018) · Mathematical Programming · doi:10.1007/s10107-018-1235-y · Cited by 348
Intelligent Reflecting Surface Enhanced Wireless Networks: Two-Timescale Beamforming Optimization
Zhao, M.-M., Wu, Q., Zhao, M.-J., & Zhang, R. (2021) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2020.3022297 · Cited by 340
Metadata
- Authors
- Hong, M., Razaviyayn, M., Luo, Z.-Q., & Pang, J.-S.
- Venue
- IEEE Signal Processing Magazine
- Published
- 2016-01
- Volume
- 33
- Issue
- 1
- Pages
- 57-77
- ISSN
- 1053-5888 (ISSN-L)
- OpenAlex
- W2210084516
- Cited by
- 447
- References
- 91
- FWCI
- 23.77
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.