The DOI matches its Crossref record; 14 of the 14 references it cites were found in a registry or an index.
Learning Shallow Neural Networks via Provable Gradient Descent with Random Initialization
Abstract
This paper presents the provable gradient descent algorithm with random initialization for learning a two-layer neural network with quadratic activation functions. Specifically, we focus on the under-parameterized regime where the number of hidden units is smaller than the dimension of the inputs. We reveal that the randomly initialized gradient descent for the nonconvex neural network training problem is able to enter a local region that enjoys strong convexity and strong smoothness within a few iterations, and then provably converges to a globally optimal model at a linear rate.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Fortunately, under the high-dimensional statistical setting, the local strong convexity and smoothness of the nonconvex loss functions can be exploited to tame the nonconvexity for various learning models, e.g., blind demixing [189], phase retrieval [254], and shallow neural networks [255].”
From Edge artificial intelligence for 6g: vision, enabling technologies, and applications (2022) · cites this paper as [255]
Citations
Cites
Identifying and attacking the saddle point problem in high-dimensional non-convex optimization
Dauphin, Y., Pascanu, R., Gulcehre, C., Cho, K., Ganguli, S., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1406.2572 · Cited by 827 · cited 1×
“, trust region method [5] and perturbed gradient descent [6].”
MethodHow to escape saddle points efficiently
Jin, C., Ge, R., Netrapalli, P., Kakade, S. M., & Jordan, M. I. (2017) · International Conference on Machine Learning · Cited by 750
Deep learning without poor local minima
Kawaguchi, K. (2016) · arXiv · doi:10.48550/arxiv.1605.07110 · Cited by 592
Theoretical Insights Into the Optimization Landscape of Over-Parameterized Shallow Neural Networks
Soltanolkotabi, M., Javanmard, A., & Lee, J. D. (2019) · IEEE Transactions on Information Theory · doi:10.1109/tit.2018.2854560 · Cited by 356
On the Computational Efficiency of Training Neural Networks
Livni, R., Shalev‐Shwartz, S., & Shamir, O. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1410.1141 · Cited by 245 · cited 2×
“In this paper, we consider a shallow neural network that consists of only one hidden layer with r neurons and activation function σ(z) = z(2) [4, 11, 12], n input nodes and one output node, as illustrated in Fig.”
MethodGradient descent with random initialization: fast global convergence for nonconvex phase retrieval
Chen, Y., Chi, Y., Fan, J., & Ma, C. (2019) · Mathematical Programming · doi:10.1007/s10107-019-01363-6 · Cited by 224 · cited 2×
“However, these algorithms have either high iteration cost or iteration complexity, yielding conservative computational guarantees for specific problems [7].”
BackgroundHow to Escape Saddle Points Efficiently
Jin, C., Ge, R., Netrapalli, P., Kakade, S. M., & Jordan, M. I. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.00887 · Cited by 205 · cited 1×
“, trust region method [5] and perturbed gradient descent [6].”
MethodA Convergence Analysis of Gradient Descent for Deep Linear Neural NetworksPreprint
Arora, S., Cohen, N., Golowich, N., & Hu, W. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.02281 · Cited by 201 · cited 2×
“Recently, the local landscape analysis turns out to be effective to enjoy fast convergence rate with cheap iteration cost via exploiting the local strong convexity and smoothness of the nonconvex loss functions [4, 8, 9, 10].”
BackgroundLi, Y., Ma, T., & Zhang, H. R. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1712.09203 · Cited by 195
Convexified Convolutional Neural NetworksPreprint
Zhang, Y., Liang, P., & Wainwright, M. J. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1609.01000 · Cited by 55 · cited 1×
“Although the nuclear norm relaxation is able to provide performance guarantees for convolutional neural networks in [2], the convex approaches are computationally expensive to deal with large-scale data set.”
Background
Research works printed but not linked: 1
- goodfellow, 2016, Deep Learning, 1
Cited by
Edge artificial intelligence for 6g: vision, enabling technologies, and applications
Letaief, K. B., Shi, Y., Lu, J., & Lu, J. (2022) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2021.3126076 · Cited by 829
“Fortunately, under the high-dimensional statistical setting, the local strong convexity and smoothness of the nonconvex loss functions can be exploited to tame the nonconvexity for various learning models, e.g., blind demixing [189], phase retrieval [254], and shallow neural networks [255].”
BackgroundSelf-citationChen, W., Yang, B., Li, J., & Wang, J. (2020) · IEEE Access · doi:10.1109/access.2020.3027794 · Cited by 95
“For example, a random initialisation gradient descent algorithm is proposed for a two-layer neural network with the quadratic activation function, which can make the network convergent to the global optimal at linear speed [30].”
BackgroundShallow hybrid quantum-classical convolutional neural network model for image classification
Wang, A., Hu, J., Zhang, S., & Li, L. (2024) · Quantum Information Processing · doi:10.1007/s11128-023-04217-5 · Cited by 57
Super-Resolution AFM Imaging Based on Enhanced Convolutional Neural Network
Han, G., Wu, T., Lv, L., Li, H., Lin, H., Lin, C., & Luo, J. (2021) · NANO · doi:10.1142/s1793292021501472 · Cited by 6
Communication-Efficient Quantized SGD for Learning Polynomial Neural Network
Yang, Z., Zhou, Y., Wu, Y., & Shi, Y. (2021) · IEEE International Performance, Computing, and Communications Conference · doi:10.1109/ipccc51483.2021.9679419 · Cited by 5
Self-citation
Metadata
- Authors
- Xia, S., & Shi, Y.
- Affiliations
- ShanghaiTech University (CN)
- Venue
- ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
- Publisher
- IEEE
- Published
- 2019-05
- Pages
- 5616-5620
- ISSN
- 1520-6149 (ISSN-L)
- OpenAlex
- W2936060459
- Cited by
- 7
- References
- 14
- Citations by year
- 2026: 1 · 2024: 2 · 2021: 3 · 2020: 1
- FWCI
- 0.38
- Type
- conference paper
- Language
- en
- Open access
- closed
- Licence
- Locations
- IEEE International Conference on Acoustics Speech and Signal Processing (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Algorithms
- Keywords
- shallow neural networks, two-layer neural networks, random initialization, gradient descent, non-convex optimization, strong convexity, linear convergence, global optimality
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Learning Shallow Neural Networks via Provable Gradient Descent with Random Initialization. https://app.cochat.ai/papers/10.1109/icassp.2019.8682855
- The paper
- Xia, S., & Shi, Y. (2019). Learning Shallow Neural Networks via Provable Gradient Descent with Random Initialization. ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). https://doi.org/10.1109/icassp.2019.8682855
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.