The DOI matches its Crossref record; 163 of the 163 references it cites were found in a registry or an index.
Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers
Abstract
Many problems of recent interest in statistics and machine learning can be posed in the framework of convex optimization. Due to the explosion in size and complexity of modern datasets, it is increasingly important to be able to solve problems with a very large number of features or training examples. As a result, both the decentralized collection or storage of these datasets as well as accompanying distributed solution methods are either necessary or at least highly desirable. Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers argues that the alternating direction method of multipliers is well suited to distributed convex optimization, and in particular to large-scale problems arising in statistics, machine learning, and related areas. The method was developed in the 1970s, with roots in the 1950s, and is equivalent or closely related to many other algorithms, such as dual decomposition, the method of multipliers, Douglas-Rachford splitting, Spingarn's method of partial inverses, Dykstra's alternating projections, Bregman iterative algorithms for ?1 problems, proximal methods, and others. After briefly surveying the theory and history of the algorithm, it discusses applications to a wide variety of statistical and machine learning problems of recent interest, including the lasso, sparse logistic regression, basis pursuit, covariance selection, support vector machines, and many others. It also discusses general distributed optimization, extensions to the nonconvex setting, and efficient implementation, including some details on distributed MPI and Hadoop MapReduce implementations.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Boyd, S., & Vandenberghe, L. (2004) · doi:10.1017/cbo9780511804441 · Cited by 56,601
Self-citationRegression shrinkage and selection via the lasso
Tibshirani, R. (1996) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1996.tb02080.x · Cited by 51,482
The Nature of Statistical Learning Theory
Vapnik, V. N. (1995) · doi:10.1007/978-1-4757-2440-0 · Cited by 39,138
Golub, G. (2012) · doi:10.56021/9781421407944 · Cited by 34,505
The Elements of Statistical Learning
Hastie, T., Tibshirani, R., & Friedman, J. (2009) · Springer Series in Statistics · doi:10.1007/978-0-387-84858-7 · Cited by 25,970
Regularization and Variable Selection Via the Elastic Net
Zou, H., & Hastie, T. (2005) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.1467-9868.2005.00503.x · Cited by 20,957
A decision-theoretic generalization of on-line learning and an application to boosting
Freund, Y., & Schapire, R. E. (1997) · Journal of Computer and System Sciences · doi:10.1006/jcss.1997.1504 · Cited by 20,466
(2005) · Statistics for Biology and Health · doi:10.1007/0-387-27255-0_9 · Cited by 20,133
MapReduce: simplified data processing on large clusters
Dean, J., & Ghemawat, S. (2008) · Communications of the ACM · doi:10.1145/1327452.1327492 · Cited by 18,476
Donoho, D. L. (2004) · Cited by 16,703
Cited by
Items 201–270 of 270
Multi-Scale Patch-Based Image Restoration
Papyan, V., & Elad, M. (2016) · IEEE Transactions on Image Processing · doi:10.1109/tip.2015.2499698 · Cited by 262
Multispectral Images Denoising by Intrinsic Tensor Sparsity Regularization
Xie, Q., Zhao, Q., Meng, D., Xu, Z., Gu, S., Zuo, W., & Zhang, L. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.187 · Cited by 261
Plug-and-Play Methods for Magnetic Resonance Imaging: Using Denoisers for Image Recovery
Ahmad, R., Bouman, C. A., Buzzard, G. T., Chan, S., Liu, S., Reehorst, E. T., & Schniter, P. (2020) · IEEE Signal Processing Magazine · doi:10.1109/msp.2019.2949470 · Cited by 261
Implicit Neural Representations with Periodic Activation Functions
Sitzmann, V., Martel, J., Bergman, A. W., Lindell, D. B., & Wetzstein, G. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.09661 · Cited by 261
A Reliable Effective Terascale Linear Learning System
Agarwal, A., Chapelle, O., Dudı́k, M., & Langford, J. C. (2011) · arXiv (Cornell University) · doi:10.48550/arxiv.1110.4198 · Cited by 260
Weighted Sparse Representation Regularized Graph Learning for RGB-T Object Tracking
Li, C., Zhao, N., Lu, Y., Zhu, C., & Tang, J. (2017) · Proceedings of the 25th ACM international conference on Multimedia · doi:10.1145/3123266.3123289 · Cited by 260
Mixed Noise Removal in Hyperspectral Image via Low-Fibered-Rank Regularization
Zheng, Y.-B., Huang, T.-Z., Zhao, X.-L., Jiang, T.-X., Ma, T.-H., & Ji, T.-Y. (2020) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2019.2940534 · Cited by 259
Hong, D., Yokoya, N., Ge, N., Chanussot, J., & Zhu, X. X. (2019) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2018.10.006 · Cited by 256
Embedded Online Optimization for Model Predictive Control at Megahertz Rates
Jerez, J. L., Goulart, P. J., Richter, S., Constantinides, G. A., Kerrigan, E. C., & Morari, M. (2014) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2014.2351991 · Cited by 256
Robust Visual Tracking Via Consistent Low-Rank Sparse Learning
Zhang, T., Liu, S., Ahuja, N., Yang, M.-H., & Ghanem, B. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-014-0738-0 · Cited by 256
An ADMM Algorithm for a Class of Total Variation Regularized Estimation Problems
Wahlberg, B., Boyd, S., Annergren, M., & Wang, Y. (2012) · IFAC Proceedings Volumes · doi:10.3182/20120711-3-be-2027.00310 · Cited by 256
Self-citationLSDT: Latent Sparse Domain Transfer Learning for Visual Adaptation
Zhang, L., Zuo, W., & Zhang, D. (2016) · IEEE Transactions on Image Processing · doi:10.1109/tip.2016.2516952 · Cited by 255
Joint Optimization of Hybrid Energy Storage and Generation Capacity With Renewable Energy
Yang, P., & Nehorai, A. (2014) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2014.2313724 · Cited by 254
Dynamic Control and Optimization of Distributed Energy Resources in a Microgrid
Wang, T., O'Neill, D., & Kamath, H. (2015) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2015.2430286 · Cited by 253
A Survey on Deep Hashing Methods
Luo, X., Wang, H., Wu, D., Chen, C., Deng, M., Huang, J., & Hua, X.-S. (2023) · ACM Transactions on Knowledge Discovery from Data · doi:10.1145/3532624 · Cited by 241
Yang, Q., Wang, H., Wang, T., Zhang, S., Wu, X., & Wang, H. (2021) · Applied Energy · doi:10.1016/j.apenergy.2021.117026 · Cited by 238
Distributed Statistical Machine Learning in Adversarial Settings: Byzantine Gradient Descent
Chen, Y., Su, L., & Xu, J. (2018) · Abstracts of the 2018 ACM International Conference on Measurement and Modeling of Computer Systems · doi:10.1145/3219617.3219655 · Cited by 228
Bringing ai to edge: from deep learning’s perspective
Liu, D., Kong, H., Luo, X., Liu, W., & Subramaniam, R. (2022) · Neurocomputing · doi:10.1016/j.neucom.2021.04.141 · Cited by 171
Learning Based Content Caching and Sharing for Wireless Networks
Song, J., Sheng, M., Quek, T. Q. S., Xu, C., & Wang, X. (2017) · IEEE Transactions on Communications · doi:10.1109/tcomm.2017.2713384 · Cited by 144
Communication-efficient distributed learning: An overview
Cao, X., Başar, T., Diggavi, S., Eldar, Y. C., Letaief, K. B., Poor, H. V., & Zhang, J. (2023) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2023.3242710 · Cited by 139
Communication Complexity of Distributed Convex Learning and Optimization
Arjevani, Y., & Shamir, O. (2015) · arXiv · doi:10.48550/arxiv.1506.01900 · Cited by 133
Linardos, A., Kushibar, K., Walsh, S., Gkontra, P., & Lekadir, K. (2022) · Scientific Reports · doi:10.1038/s41598-022-07186-4 · Cited by 130
Deep Learning for Distributed Optimization: Applications to Wireless Resource Management
Lee, H., Lee, S. H., & Quek, T. Q. S. (2019) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2019.2933890 · Cited by 129
Federated Tensor Factorization for Computational Phenotyping
Kim, Y., Sun, J., Yu, H., & Jiang, X. (2017) · Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/3097983.3098118 · Cited by 128
Linear Convergence Rate of a Class of Distributed Augmented Lagrangian Algorithms
Jakovetic, D., Moura, J. M. F., & Xavier, J. (2015) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2014.2363299 · Cited by 123
Modality-correlation-aware sparse representation for RGB-infrared object tracking
Lan, X., Ye, M., Zhang, S., Zhou, H., & Yuen, P. C. (2020) · Pattern Recognition Letters · doi:10.1016/j.patrec.2018.10.002 · Cited by 121
Distributed stochastic optimization and learning
Shamir, O., & Srebro, N. (2014) · 2014 52nd Annual Allerton Conference on Communication, Control, and Computing (Allerton) · doi:10.1109/allerton.2014.7028543 · Cited by 120
Large-Scale Convex Optimization for Dense Wireless Cooperative Networks
Shi, Y., Zhang, J., O'Donoghue, B., & Letaief, K. B. (2015) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2015.2443731 · Cited by 118
Partial Occlusion Handling for Visual Tracking via Robust Part Matching
Zhang, T., Jia, K., Xu, C., Ma, Y., & Ahuja, N. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.164 · Cited by 117
Large-scale convex optimization for ultra-dense cloud-RAN
Shi, Y., Zhang, J., Letaief, K. B., Bai, B., & Chen, W. (2015) · IEEE Wireless Communications · doi:10.1109/mwc.2015.7143330 · Cited by 108
Loss-aware Weight Quantization of Deep Networks
Hou, L., & Kwok, J. T. (2018) · arXiv · doi:10.48550/arxiv.1802.08635 · Cited by 99
Distributed Coordination of Charging Stations Considering Aggregate EV Power Flexibility
Yan, D., Ma, C., & Chen, Y. (2023) · IEEE Transactions on Sustainable Energy · doi:10.1109/tste.2022.3213173 · Cited by 96
Turning a denoiser into a super-resolver using plug and play priors
Brifman, A., Romano, Y., & Elad, M. (2016) · 2016 IEEE International Conference on Image Processing (ICIP) · doi:10.1109/icip.2016.7532589 · Cited by 91
Sohrabpour, A., Lu, Y., Worrell, G., & He, B. (2016) · NeuroImage · doi:10.1016/j.neuroimage.2016.05.064 · Cited by 89
Imaging neural activity in the ventral nerve cord of behaving adult Drosophila
Chen, C.-L., Hermans, L., Viswanathan, M. C., Fortun, D., Aymanns, F., Unser, M., Cammarato, A., Dickinson, M. H., & Ramdya, P. (2018) · Nature Communications · doi:10.1038/s41467-018-06857-z · Cited by 84
Exploiting Unlabeled Data in Smart Cities using Federated Edge Learning
Albaseer, A., Ciftler, B. S., Abdallah, M., & Al-Fuqaha, A. (2020) · 2020 International Wireless Communications and Mobile Computing (IWCMC) · doi:10.1109/iwcmc48107.2020.9148475 · Cited by 84
Towards Data Poisoning Attacks in Crowd Sensing Systems
Miao, C., Li, Q., Xiao, H., Jiang, W., Huai, M., & Su, L. (2018) · Proceedings of the Eighteenth ACM International Symposium on Mobile Ad Hoc Networking and Computing · doi:10.1145/3209582.3209594 · Cited by 77
Gossip training for deep learning
Blot, M., Picard, D., Cord, M., & Thome, N. (2016) · arXiv · doi:10.48550/arxiv.1611.09726 · Cited by 77
Robust and communication-efficient collaborative learning
Reisizadeh, A., Taheri, H., Mokhtari, A., Hassani, H., & Pedarsani, R. (2019) · arXiv · doi:10.48550/arxiv.1907.10595 · Cited by 73
GLMix: Generalized Linear Mixed Models For Large-Scale Response Prediction
Zhang, X., Zhou, Y., Ma, Y., Chen, B.-C., Zhang, L., & Agarwal, D. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939684 · Cited by 71
Consistent sparsification for graph optimization
Huang, G., Kaess, M., & Leonard, J. J. (2013) · 2013 European Conference on Mobile Robots · doi:10.1109/ecmr.2013.6698835 · Cited by 70
Smoothed -Minimization for Green Cloud-RAN With User Admission Control
Shi, Y., Cheng, J., Zhang, J., Bai, B., Chen, W., & Letaief, K. B. (2016) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2016.2544578 · Cited by 70
Rodosthenous, T., Shahrezaei, V., & Evangelou, M. (2020) · Bioinformatics · doi:10.1093/bioinformatics/btaa530 · Cited by 67
Robust Group Sparse Beamforming for Multicast Green Cloud-RAN With Imperfect CSI
Shi, Y., Zhang, J., & Letaief, K. B. (2015) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2015.2442957 · Cited by 63
Nonlinear factor recovery for long-term SLAM
Mazuran, M., Burgard, W., & Tipaldi, G. D. (2016) · The International Journal of Robotics Research · doi:10.1177/0278364915581629 · Cited by 63
Hierarchical distributed framework for EV charging scheduling using exchange problem
Khaki, B., Chu, C., & Gadh, R. (2019) · Applied Energy · doi:10.1016/j.apenergy.2019.03.008 · Cited by 62
Jiang, T., Shi, Y., Zhang, J., & Letaief, K. B. (2019) · IEEE Internet of Things Journal · doi:10.1109/jiot.2018.2881486 · Cited by 60
Froelicher, D., Troncoso-Pastoriza, J. R., Sousa, J. S., & Hubaux, J.-P. (2020) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2020.2976612 · Cited by 59
EasiEdge: A Novel Global Deep Neural Networks Pruning Method for Efficient Edge Computing
Yu, F., Cui, L., Wang, P., Han, C., Huang, R., & Huang, X. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.3034925 · Cited by 58
Moiré pattern removal from texture images via low-rank and sparse matrix decomposition
Liu, F., Yang, J., & Yue, H. (2015) · 2015 Visual Communications and Image Processing (VCIP) · doi:10.1109/vcip.2015.7457907 · Cited by 56
A two-stage autonomous EV charging coordination method enabled by blockchain
Ping, J., Yan, Z., & Chen, S. (2021) · Journal of Modern Power Systems and Clean Energy · doi:10.35833/mpce.2019.000139 · Cited by 55
Applications of Metaheuristics in Reservoir Computing Techniques: A Review
Bala, A., Ismail, I., Ibrahim, R., & Sait, S. M. (2018) · IEEE Access · doi:10.1109/access.2018.2873770 · Cited by 53
SP-CIDS: Secure and Private Collaborative IDS for VANETs
Raja, G., Anbalagan, S., Vijayaraghavan, G., Theerthagiri, S., Suryanarayan, S. V., & Wu, X.-W. (2021) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2020.3036071 · Cited by 53
Improving quality of data: IoT data aggregation using device-to-device communications
Sanyal, S., & Zhang, P. (2018) · IEEE Access · doi:10.1109/access.2018.2878640 · Cited by 51
Supervised Learning Under Distributed Features
Ying, B., Yuan, K., & Sayed, A. H. (2019) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2018.2881661 · Cited by 50
Beyond class-level privacy leakage: breaking record-level privacy in federated learning
Yuan, X., Ma, X., Zhang, L., Fang, Y., & Wu, D. (2022) · IEEE Internet of Things Journal · doi:10.1109/jiot.2021.3089713 · Cited by 45
Burn scar detection using cloudy MODIS images via low-rank and sparsity-based models
Dao, M., Kwan, C., Ayhan, B., & Tran, T. D. (2016) · 2016 IEEE Global Conference on Signal and Information Processing (GlobalSIP) · doi:10.1109/globalsip.2016.7905827 · Cited by 42
Secure Architectures Implementing Trusted Coalitions for Blockchained Distributed Learning (TCLearn)
Lugan, S., Desbordes, P., Brion, E., Ramos Tormo, L. X., Legay, A., & Macq, B. (2019) · IEEE Access · doi:10.1109/access.2019.2959220 · Cited by 39
Learning Privately over Distributed Features: An ADMM Sharing Approach
Hu, Y., Liu, P., Kong, L., & Niu, D. (2019) · arXiv · doi:10.48550/arxiv.1907.07735 · Cited by 30
On the Linear Convergence of Distributed Optimization over Directed Graphs
Xi, C., & Khan, U. A. (2015) · arXiv · doi:10.48550/arxiv.1510.02149 · Cited by 29
Quantum Combinatorial Optimization in the NISQ Era: A Systematic Mapping Study
Gemeinhardt, F., Garmendia, A., Wimmer, M., Weder, B., & Leymann, F. (2024) · ACM Computing Surveys · doi:10.1145/3620668 · Cited by 22
Harnessing wireless channels for scalable and privacy-preserving federated learning
Elgabli, A., Park, J., Issaid, C. B., & Bennis, M. (2021) · IEEE Transactions on Communications · doi:10.1109/tcomm.2021.3078783 · Cited by 14
Massive CSI Acquisition for Dense Cloud-RANs With Spatial-Temporal Dynamics
Liu, X., Shi, Y., Zhang, J., & Letaief, K. B. (2018) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2018.2797969 · Cited by 11
Iutzeler, F., Bianchi, P., Ciblat, P., & Hachem, W. (2014) · 53rd IEEE Conference on Decision and Control · doi:10.1109/cdc.2014.7040177 · Cited by 10
Understanding and Predicting Weight Loss with Mobile Social Networking Data
Wang, Z., Derr, T., Yin, D., & Tang, J. (2017) · Proceedings of the 2017 ACM on Conference on Information and Knowledge Management · doi:10.1145/3132847.3133019 · Cited by 9
Federated Learning for Multi-Center Imaging Diagnostics: A Study in Cardiovascular Disease
Linardos, A., Kushibar, K., Walsh, S., Gkontra, P., & Lekadir, K. (2021) · doi:10.21203/rs.3.rs-688924/v1 · Cited by 9
Stochastic proximal gradient consensus over time-varying networks
Hong, M., & Chang, T.-H. (2016) · 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2016.7472584 · Cited by 6
Fused graphical lasso recovers flowering time mutation genes in arabidopsis thaliana
Kapoor, R., Datta, A., & Thomson, M. (2021) · Inventions · doi:10.3390/inventions6030052 · Cited by 5
A new multiple kernel-based regularization method for identification of delay linear dynamic systems
Chen, X., Mao, Z., & Jia, R. (2020) · Chemometrics and Intelligent Laboratory Systems · doi:10.1016/j.chemolab.2020.103971 · Cited by 5
Augmented distributed optimization for networked systems
Xu, J. (2019) · doi:10.32657/10356/68805 · Cited by 5
Metadata
- Authors
- Boyd, S.
- Venue
- now publishers, Inc. eBooks
- Publisher
- now Publishers Inc
- Published
- 2010
- OpenAlex
- W2164278908
- Cited by
- 12,677
- References
- 163
- FWCI
- 142.38
- Type
- book
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.