Mathematical Models of Overparameterized Neural Networks
Open accessThe DOI matches its Crossref record; 122 of the 134 references it cites were found in a registry or an index.
Abstract
Deep learning has received considerable empirical success in recent years. However, while many ad hoc tricks have been discovered by practitioners, until recently, there has been a lack of theoretical understanding for tricks invented in the deep learning literature. Known by practitioners that overparameterized neural networks (NNs) are easy to learn, in the past few years, there have been important theoretical developments in the analysis of overparameterized NNs. In particular, it was shown that such systems behave like convex systems under various restricted settings, such as for two-layer NNs, and when learning is restricted locally in the so-called neural tangent kernel space around specialized initializations. This article discusses some of these recent signs of progress leading to a significantly better understanding of NNs. We will focus on the analysis of two-layer NNs and explain the key mathematical models, with their algorithmic implications. We will then discuss challenges in understanding deep NNs and some current research directions.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures













Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Besides, channel perturbation in algorithm iterates can also serve as the mechanism to design saddle points escaping algorithms [94], thereby establishing global optimality for training the non-convex over-parameterized neural networks in high-dimensional statistical settings [180].”
From Edge artificial intelligence for 6g: vision, enabling technologies, and applications (2022) · cites this paper as [180]
Experimental — accuracy not yet measuredIn the citing paper
“However, empirical risk minimization for training the models is usually nonconvex, which poses significant challenges to guarantee global optimality for the learning algorithms and models [180].”
From Edge artificial intelligence for 6g: vision, enabling technologies, and applications (2022) · cites this paper as [180]
In the citing paper
“Besides, with high-dimensional datasets, the nonconvex loss functions of certain statistical learning models, including over-parameterized neural networks [180] and dictionary learning [256], can enjoy benign global geometric landscape such that all the local minima are global minima, and all the saddle points can be escaped efficiently using the algorithms including trust region method and pertur”
From Edge artificial intelligence for 6g: vision, enabling technologies, and applications (2022) · cites this paper as [180]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 1×
“3048020 deep NNs (DNNs) have largely improved the state-of-theart performances in numerous real applications, such as image classification [1], [2], speech recognition [3], and natural language processing [4].”
BackgroundImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · cited 1×
“3048020 deep NNs (DNNs) have largely improved the state-of-theart performances in numerous real applications, such as image classification [1], [2], speech recognition [3], and natural language processing [4].”
BackgroundGradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805 · cited 1×
“com/exdb/mnist/ [42] to demonstrate various aspects of the different frameworks.”
Background(2014) · Cited by 33,419
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818
Auto-Encoding Variational Bayes
Kingma, D. P., & Welling, M. (2013) · arXiv · doi:10.48550/arxiv.1312.6114 · Cited by 20,067
Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.123 · Cited by 19,741 · cited 2×
“In real applications, initialization strategies, e.g., [74], [45] sample the NN weights from N (0 , O ( m )) , with variance approaching infinity as the number of hidden nodes m goes to ∞ .”
BackgroundMethodEfficientNet: Rethinking Model Scaling for Convolutional Neural Networks
Tan, M., & Le, Q. V. (2019) · arXiv · doi:10.48550/arxiv.1905.11946 · Cited by 17,072 · cited 1×
“It was observed, in practice, that it is beneficial to increase both depth and width simultaneously to balance the tradeoff [90].”
BackgroundNeural Machine Translation by Jointly Learning to Align and Translate
Bahdanau, D., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1409.0473 · Cited by 16,213 · cited 1×
“3048020 deep NNs (DNNs) have largely improved the state-of-theart performances in numerous real applications, such as image classification [1], [2], speech recognition [3], and natural language processing [4].”
Background(2010) · Cited by 14,134
Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond
Schölkopf, B., & Smola, A. J. (2001) · The MIT Press eBooks · doi:10.7551/mitpress/4175.001.0001 · Cited by 11,703
Hinton, G., Deng, L., Yu, D., Dahl, G., Mohamed, A.-R., Jaitly, N., Senior, A., Vanhoucke, V., Nguyen, P., Sainath, T., et al. (2012) · IEEE Signal Processing Magazine · doi:10.1109/msp.2012.2205597 · Cited by 10,260
Dropout as a bayesian approximation: Representing model uncertainty in deep learning
Gal, Y., & Ghahramani, Z. (2015) · arXiv · doi:10.48550/arxiv.1506.02142 · Cited by 6,471 · cited 1×
“For example, it was argued in [108] using the Gaussian process point of view that dropout [5] can be used to obtain uncertainty estimation for NNs.”
Bayesian Learning for Neural Networks
Neal, R. M. (1996) · Lecture Notes in Statistics · doi:10.1007/978-1-4612-0745-0 · Cited by 4,276 · cited 3×
“The earliest study of overparameterized infinite-width NN was motivated by this Bayesian interpretation [8], [9], [106], where the relationship of infinite-width NN and Gaussian processes was investigated, which is only based on the random feature.”
BackgroundUnderstanding deep learning requires rethinking generalization
Zhang, C., Bengio, S., Hardt, M., Recht, B., & Vinyals, O. (2016) · arXiv · doi:10.48550/arxiv.1611.03530 · Cited by 2,838 · cited 1×
“In addition to the above empirical observations, it is also known by practitioners that overparameterized neural networks (NNs) with many hidden units are easy to learn [7].”
BackgroundUniversal approximation bounds for superpositions of a sigmoidal function
Barron, A. R. (1993) · IEEE Transactions on Information Theory · doi:10.1109/18.256500 · Cited by 2,804 · cited 1×
“It is well known that two-layer NNs are universal approximators [84], [85].”
Background(2008) · doi:10.1007/978-3-7643-8722-8 · Cited by 2,220
Leshno, M., Lin, V. Y., Pinkus, A., & Schocken, S. (1993) · Neural Networks · doi:10.1016/s0893-6080(05)80131-5 · Cited by 2,121 · cited 1×
“It is well known that two-layer NNs are universal approximators [84], [85].”
BackgroundRepresentation of Events in Nerve Nets and Finite Automata
Kleene, S. C. (1956) · Automata Studies. (AM-34) · doi:10.1515/9781400882618-002 · Cited by 1,655 · cited 1×
“Two-layer NNs have a history dating back to the 1940s [11].”
BackgroundNeural Tangent Kernel: Convergence and Generalization in Neural Networks
Jacot, A., Gabriel, F., & Hongler, C. (2018) · arXiv · doi:10.48550/arxiv.1806.07572 · Cited by 1,562 · cited 2×
“Such a generalization leads to NTK [17], which we shall describe in this section.”
BackgroundVisualizing the Loss Landscape of Neural Nets
Li, H., Xu, Z., Taylor, G., Studer, C., & Goldstein, T. (2017) · Repository for Publications and Research Data (ETH Zurich) · doi:10.3929/ethz-b-000461393 · Cited by 926
The Expressive Power of Neural Networks: A View from the WidthPreprint
Zhou, L., Pu, H., Wang, F., Hu, Z., & Wang, L. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1709.02540 · Cited by 864 · cited 1×
“Related results show that DNNs can represent any function with a constant number of nodes per layer [88], [89], which suggest a tradeoff between the depth and the width in terms of universal approximation.”
BackgroundHow to escape saddle points efficiently
Jin, C., Ge, R., Netrapalli, P., Kakade, S. M., & Jordan, M. I. (2017) · International Conference on Machine Learning · Cited by 750
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.1.1 · Cited by 676 · cited 1×
“For example, several researchers observed that NN’s generalization ability is related to the sharpness/flatness of the local minimal solution resulted from training and discussed different methods to characterize flatness [91]–[93].”
BackgroundA Convergence Theory for Deep Learning via Over-Parameterization
Allen-Zhu, Z., Li, Y., & Song, Z. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.03962 · Cited by 657
Deep learning without poor local minima
Kawaguchi, K. (2016) · arXiv · doi:10.48550/arxiv.1605.07110 · Cited by 592
Deep double descent: where bigger models and more data hurt*
Nakkiran, P., Kaplun, G., Bansal, Y., Yang, T., Barak, B., & Sutskever, I. (2021) · Journal of Statistical Mechanics: Theory and Experiment · doi:10.1088/1742-5468/ac3a74 · Cited by 565
Weighted Sums of Random Kitchen Sinks: Replacing minimization with randomization in learning
Rahimi, A., & Recht, B. (2008) · Cited by 558
A Theoretical Analysis of Deep Q-LearningPreprint
Fan, J., Wang, Z., Xie, Y., & Yang, Z. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.00137 · Cited by 548 · cited 1×
“, [20], [18], [21], [22], [23], [24], [25], [26], [27], [28], [29], [30], [31], [32], [33], [34], [35], [36], [37] and sharper generalization errors, e.”
Deep Neural Networks as Gaussian Processes
Lee, J., Bahri, Y., Novak, R., Schoenholz, S. S., Pennington, J., & Sohl‐Dickstein, J. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.00165 · Cited by 529 · cited 1×
“The earliest study of overparameterized infinite-width NN was motivated by this Bayesian interpretation [8], [9], [106], where the relationship of infinite-width NN and Gaussian processes was investigated, which is only based on the random feature.”
BackgroundOn Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima
Keskar, N. S., Mudigere, D., Nocedal, J., Smelyanskiy, M., & Tang, P. T. P. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1609.04836 · Cited by 525 · cited 1×
“For example, several researchers observed that NN’s generalization ability is related to the sharpness/flatness of the local minimal solution resulted from training, and discussed different methods to characterize flatness [91], [92], [93].”
BackgroundA mean field view of the landscape of two-layer neural networks
Mei, S., Montanari, A., & Nguyen, P.-M. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1806579115 · Cited by 500 · cited 4×
“Moreover, under mild conditions, we have ρt(u, θ) converges (weakly) to a distribution ρ∞(u, θ) that can be lower bounded by a normal distribution [46].”
BackgroundMethodInfluentialStatistical Mechanics of Learning
Engel, A., & Van den Broeck, C. (2001) · Cambridge University Press eBooks · doi:10.1017/cbo9781139164542 · Cited by 483 · cited 1×
“The idea of studying the mean filed limit comes from statistical physics [55], which suggests that the mathematical model of a large number of interacting neurons can be simplified using the probability distribution that represents the average effect.”
BackgroundMean Field Analysis of Neural Networks: A Law of Large Numbers
Sirignano, J. A., & Spiliopoulos, K. (2019) · SIAM Journal on Applied Mathematics · doi:10.1016/j.spa.2019.06.003 · Cited by 430 · cited 1×
“In this section, we introduce another line of research, which applies the mathematical tools of the meanfield analysis from statistical physics to study two-layer neural networks [46], [47], [48], [49], [50], [51], [52], [51], [53], [54].”
MethodPrevalence of neural collapse during the terminal phase of deep learning training
Papyan, V., Han, X. Y., & Donoho, D. L. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2015509117 · Cited by 403 · cited 1×
“As another example, [113] observed a phenomenon called Neural Collapse , which states that predicted class means collapse to the vertices of a Simplex Equiangular Tight Frame at the final training stage.”
BackgroundImplicit Bias of Gradient Descent for Wide Two-layer Neural Networks Trained with the Logistic LossPreprint
Chizat, L., & Bach, F. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.04486 · Cited by 364 · cited 2×
“There were a number of recent works that attempted to generalize MF [57], [66]–[71].”
BackgroundResultOn Exact Computation with an Infinitely Wide Neural Net
Arora, S., Du, S. S., Hu, W. P., Li, Z., Salakhutdinov, R., & Wang, R. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.11955 · Cited by 358 · cited 2×
“…years, in which a number of researchers proved polynomial convergence rates to the global optimal, e.g., [20], [18], [21], [22], [23], [24], [25], [26], [27], [28], [29], [30], [31], [32], [33], [34], [35], [36], [37] and sharper generalization errors, e.g., [20], [23], [21], [38], [39], [40].”
MethodWide neural networks of any depth evolve as linear models under gradient descent *
Lee, J., Xiao, L., Schoenholz, S. S., Bahri, Y., Novak, R., Sohl-Dickstein, J., & Pennington, J. (2020) · Journal of Statistical Mechanics: Theory and Experiment · doi:10.1088/1742-5468/abc62b · Cited by 353 · cited 1×
“More recently, some articles also investigate the kernel form of the NTK regime to perform the Bayesian inference [107], which has a larger function class than NF.”
BackgroundGradient Descent Provably Optimizes Over-parameterized Neural Networks
Du, S. S., Zhai, X., Póczos, B., & Singh, A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.02054 · Cited by 344 · cited 3×
“For two-layer NNs, for example, by assuming that the minimum eigenvalue of the kernel matrix for the training data is positive, denoted as λ0, it was shown in [41] that, when the number of hidden units is greater than n6λ−4 0 δ −2, with a learning rate of η = O(λ0n−2), then, with probability at least 1 − δ, the GD met…”
BackgroundMethodBenefits of Depth in Neural Networks
Telgarsky, M. (2016) · arXiv · doi:10.48550/arxiv.1602.04485 · Cited by 341 · cited 1×
“However, for certain functions that can be represented by deep neural networks with a small number of nodes, exponentially number of nodes are needed to represent them with shallow neural networks [86], [87].”
BackgroundOn the Global Convergence of Gradient Descent for Over-parameterized Models using Optimal TransportPreprint
Chizat, L., & Bach, F. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.09545 · Cited by 334 · cited 4×
“More precisely, the corresponding particle movement in the continuous limit obeys the gradient flow equation [47]”
BackgroundMethodResultInfluentialIdentity Matters in Deep LearningPreprint
Hardt, M., & Ma, T. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.04231 · Cited by 333 · cited 1×
“With the help of restrictive assumptions, or for specialized models, a number of earlier works, e.g., [96], [97], [98], [99], [100], [101], [102], [103], studied the theoretical characterizations of NN landscapes.”
BackgroundOn the Power of Over-parametrization in Neural Networks with Quadratic ActivationPreprint
Du, S. S., & Lee, J. D. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.01206 · Cited by 287
Mean-field theory of two-layers neural networks: dimension-free bounds and kernel limitPreprint
Song, M. N., Misiakiewicz, T., & Montanari, A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.06015 · Cited by 279 · cited 3×
“, [20], [18], [21], [22], [23], [24], [25], [26], [27], [28], [29], [30], [31], [32], [33], [34], [35], [36], [37] and sharper generalization errors, e.”
BackgroundUniversal Function Approximation by Deep Neural Nets with Bounded Width and ReLU Activations
Hanin, B. (n.d.) · RePEc: Research Papers in Economics · Cited by 278
Learning Overparameterized Neural Networks via Stochastic Gradient Descent on Structured Data
Li, Y., & Liang, Y. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1808.01204 · Cited by 257 · cited 3×
“, [18] and [20]–[37], and sharper generalization errors, e.”
BackgroundMethodNeural Policy Gradient Methods: Global Optimality and Rates of ConvergencePreprint
Wang, L., Cai, Q., Yang, Z., & Wang, Z. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.01150 · Cited by 248 · cited 1×
“…years, in which a number of researchers proved polynomial convergence rates to the global optimal, e.g., [20], [18], [21], [22], [23], [24], [25], [26], [27], [28], [29], [30], [31], [32], [33], [34], [35], [36], [37] and sharper generalization errors, e.g., [20], [23], [21], [38], [39], [40].”
Lower bounds for non-convex stochastic optimization
Arjevani, Y., Carmon, Y., Duchi, J. C., Foster, D. J., Srebro, N., & Woodworth, B. (2022) · Mathematical Programming · doi:10.1007/s10107-022-01822-7 · Cited by 244 · cited 1×
“This question was resolved satisfactorily for general nonconvex problems, where both the convergence rate of SGD and that of the optimal stochastic algorithm were known [82], [83], [105].”
BackgroundUniform approximation of functions with random bases
Rahimi, A., & Recht, B. (2008) · 2008 46th Annual Allerton Conference on Communication, Control, and Computing · doi:10.1109/allerton.2008.4797607 · Cited by 238 · cited 2×
“The random feature approach (denoted by RF in this article) is also referred to as random kitchen sinks [10], [12], [13].”
BackgroundAn Improved Analysis of Training Over-parameterized Deep Neural NetworksPreprint
Zou, D., & Gu, Q. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.04688 · Cited by 237 · cited 1×
“, [20], [18], [21], [22], [23], [24], [25], [26], [27], [28], [29], [30], [31], [32], [33], [34], [35], [36], [37] and sharper generalization errors, e.”
On Lazy Training in Differentiable Programming
Chizat, L., Oyallon, E., & Bach, F. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.07956 · Cited by 235 · cited 1×
“This phenomenon is also referred to as the “lazy training” regime in [19].”
BackgroundSPIDER: Near-Optimal Non-Convex Optimization via Stochastic Path Integrated Differential Estimator
Fang, C., Li, C. J., Lin, Z., & Zhang, T. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.01695 · Cited by 226 · cited 1×
“This question was resolved satisfactorily for general non-convex problems, where both the convergence rate of SGD and that of the optimal stochastic algorithm were known [82], [83], [105].”
BackgroundSelf-citationThree Factors Influencing Minima in SGD
Jastrzȩbski, S., Kenton, Z., Arpit, D., Ballas, N., Fischer, A., Bengio, Y., & Storkey, A. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.04623 · Cited by 217 · cited 1×
“A few other works have also mentioned the phenomenon that a large initial learning rate leads to better NN solution [43], [44], which cannot be explained by the NTK view very well.”
BackgroundNeural Proximal/Trust Region Policy Optimization Attains Globally Optimal PolicyPreprint
Liu, B., Cai, Q., Yang, Z., & Wang, Z. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.10306 · Cited by 213 · cited 1×
“…years, in which a number of researchers proved polynomial convergence rates to the global optimal, e.g., [20], [18], [21], [22], [23], [24], [25], [26], [27], [28], [29], [30], [31], [32], [33], [34], [35], [36], [37] and sharper generalization errors, e.g., [20], [23], [21], [38], [39], [40].”
Arora, S., Du, S. S., Hu, W. P., Li, Z., & Wang, R. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.08584 · Cited by 210 · cited 2×
“…years, in which a number of researchers proved polynomial convergence rates to the global optimal, e.g., [20], [18], [21], [22], [23], [24], [25], [26], [27], [28], [29], [30], [31], [32], [33], [34], [35], [36], [37] and sharper generalization errors, e.g., [20], [23], [21], [38], [39], [40].”
How to Escape Saddle Points Efficiently
Jin, C., Ge, R., Netrapalli, P., Kakade, S. M., & Jordan, M. I. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.00887 · Cited by 205 · cited 2×
“…landscape of NNs. Related studies of neural network training were investigated from the generic non-convex optimization point of view, where a main issue was the complexity of stochastic optimization algorithms such as SGD to escape saddle points and converge to local minimal solutions [80], [81].”
BackgroundUniversal Function Approximation by Deep Neural Nets with Bounded Width and ReLU Activations
Hanin, B. (2019) · Mathematics · doi:10.3390/math7100992 · Cited by 202 · cited 1×
“Related results show that DNNs can represent any function with a constant number of nodes per layer [88], [89], which suggest a tradeoff between the depth and the width in terms of universal approximation.”
BackgroundA Theoretical Analysis of Deep Q-Learning
Yang, Z., Xie, Y., & Wang, Z. (2020) · Learning for Dynamics and Control · Cited by 200
On the Power and Limitations of Random Features for Understanding Neural NetworksPreprint
Yehudai, G., & Shamir, O. (2019) · Neural Information Processing Systems · doi:10.48550/arxiv.1904.00687 · Cited by 198 · cited 2×
“Notably, Yehudai and Shamir [63] showed that random features cannot be used to learn even a single ReLU neuron unless the number of the hidden units is exponentially large in d.”
BackgroundTopology and Geometry of Half-Rectified Network OptimizationPreprint
Freeman, C. D., & Bruna, J. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.01540 · Cited by 194 · cited 1×
“, [96], [97], [98], [99], [100], [101], [102], [103], studied the theoretical characterizations of NN landscapes.”
Gradient Descent Finds Global Minima of Deep Neural Networks
Du, S. S., Lee, J. D., Li, H., Wang, L., & Zhai, X. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.03804 · Cited by 189 · cited 3×
“, [18] and [20]–[37], and sharper generalization errors, e.”
BackgroundComputing with Infinite Networks
Williams, C. K. I. (1996) · Aston Publications Explorer (Aston University) · Cited by 187
Finite Depth and Width Corrections to the Neural Tangent KernelPreprint
Hanin, B., & Nica, M. (2019) · International Conference on Learning Representations · doi:10.48550/arxiv.1909.05989 · Cited by 185 · cited 1×
“, Allen-Zhu and Li [21], Hanin and Nica [76], Bai and Lee [77], and Huang and Yau [78], have tried to model NNs beyond a linear approximation of NTK, typically second-order approximation.”
BackgroundOn the Equivalence between Kernel Quadrature Rules and Random Feature ExpansionsPreprint
Bach, F. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1502.06800 · Cited by 183 · cited 1×
“The random feature approach (denoted by RF in this article) is also referred to as random kitchen sinks [10], [12], [13].”
BackgroundDaniely, A., Frostig, R., & Singer, Y. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1602.05897 · Cited by 178 · cited 1×
“One interesting extension of the RF theory that can be used to analyze the behavior of NN is presented in [15] and [16].”
BackgroundMean-field Langevin dynamics and energy landscape of neural networks
Hu, K., Ren, Z., Šiška, D., & Szpruch, Ł. (2021) · Annales de l Institut Henri Poincaré Probabilités et Statistiques · doi:10.1214/20-aihp1140 · Cited by 177 · cited 1×
“In this section, we introduce another line of research, which applies the mathematical tools of the mean-field analysis from statistical physics to study two-layer neural networks [46], [47], [48], [49], [50], [51], [52], [51], [53], [54].”
MethodStochastic Gradient Descent Optimizes Over-parameterized Deep ReLU Networks
Zou, D., Cao, Y. J., Zhou, D., & Gu, Q. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.08888 · Cited by 175 · cited 2×
“, [18], [22], and [24], and the technique can also be generalized to handle more complex topological structures, such as recurrent NNs [21] and residual NNs [18].”
BackgroundMethodEscaping From Saddle Points --- Online Stochastic Gradient for Tensor Decomposition
Ge, R., Huang, F., Jin, C., & Yang, Y. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1503.02101 · Cited by 173 · cited 2×
“Related studies of NN training were investigated from the generic nonconvex optimization point of view, where the main issue was the complexity of stochastic optimization algorithms, such as SGD to escape saddle points and converge to local minimal solutions [80], [81].”
BackgroundThe loss surface of deep and wide neural networks
Nguyen, Q., & Hein, M. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1704.08045 · Cited by 156 · cited 1×
“With the help of restrictive assumptions, or for specialized models, a number of earlier works, e.g., [96], [97], [98], [99], [100], [101], [102], [103], studied the theoretical characterizations of NN landscapes.”
BackgroundThe Full Spectrum of Deepnet Hessians at Scale: Dynamics with SGD Training and Sample SizePreprint
Papyan, V. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.07062 · Cited by 152
Learning and Generalization in Overparameterized Neural Networks, Going Beyond Two Layers
Allen-Zhu, Z., Li, Y., & Liang, Y. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.04918 · Cited by 151
Generalization Bounds of Stochastic Gradient Descent for Wide and Deep Neural Networks
Cao, Y. J., & Gu, Q. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.13210 · Cited by 151 · cited 1×
“…years, in which a number of researchers proved polynomial convergence rates to the global optimal, e.g., [20], [18], [21], [22], [23], [24], [25], [26], [27], [28], [29], [30], [31], [32], [33], [34], [35], [36], [37] and sharper generalization errors, e.g., [20], [23], [21], [38], [39], [40].”
Yang, G. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.04760 · Cited by 150 · cited 1×
“…years, in which a number of researchers proved polynomial convergence rates to the global optimal, e.g., [20], [18], [21], [22], [23], [24], [25], [26], [27], [28], [29], [30], [31], [32], [33], [34], [35], [36], [37] and sharper generalization errors, e.g., [20], [23], [21], [38], [39], [40].”
Limitations of Lazy Training of Two-layers Neural NetworksPreprint
Ghorbani, B., Song, M. N., Misiakiewicz, T., & Andrea Montanari, A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.08899 · Cited by 147
An Investigation into Neural Net Optimization via Hessian Eigenvalue Density
Ghorbani, B., Krishnan, S., & Xiao, Y. (2019) · arXiv · doi:10.48550/arxiv.1901.10159 · Cited by 144 · cited 1×
“, [94] and [95], that attempted to understand the Hessian matrices of NNs.”
BackgroundLinearized two-layers neural networks in high dimension
Ghorbani, B., Mei, S., Misiakiewicz, T., & Montanari, A. (2021) · The Annals of Statistics · doi:10.1214/20-aos1990 · Cited by 142
Why Deep Neural Networks for Function Approximation?
Liang, S., & Srikant, R. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1610.04161 · Cited by 135
The Global Landscape of Neural Networks: An Overview
Sun, R., Li, D., Liang, S., Ding, T., & Srikant, R. (2020) · IEEE Signal Processing Magazine · doi:10.1109/msp.2020.3004124 · Cited by 134 · cited 1×
“We also refer the readers to the review [104] and the references therein for the global landscape of NNs.”
BackgroundNo bad local minima: Data independent training error guarantees for multilayer neural networks
Soudry, D., & Carmon, Y. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.08361 · Cited by 133 · cited 1×
“With the help of restrictive assumptions, or for specialized models, a number of earlier works, e.g., [96], [97], [98], [99], [100], [101], [102], [103], studied the theoretical characterizations of NN landscapes.”
BackgroundSpurious Local Minima are Common in Two-Layer ReLU Neural Networks
Safran, I., & Shamir, O. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1712.08968 · Cited by 133
A rigorous framework for the mean field limit of multilayer neural networks
Nguyen, P.-M., & Pham, H. T. (2023) · Mathematical Statistics and Learning · doi:10.4171/msl/42 · Cited by 131
When do neural networks outperform kernel methods?*
Ghorbani, B., Mei, S., Misiakiewicz, T., & Montanari, A. (2021) · Journal of Statistical Mechanics Theory and Experiment · doi:10.1088/1742-5468/ac3a81 · Cited by 129 · cited 1×
“to investigate the difference between the lazy training condition in NTK and the actual training process of NNs [52], [58], [62]–[65].”
BackgroundBackward Feature Correction: How Deep Learning Performs Deep LearningPreprint
Allen-Zhu, Z., & Li, Y. (2020) · arXiv.org · doi:10.48550/arxiv.2001.04413 · Cited by 127
Sharp Analysis for Nonconvex SGD Escaping from Saddle PointsPreprint
Fang, C., Lin, Z., & Zhang, T. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.00247 · Cited by 115 · cited 1×
“This question was resolved satisfactorily for general nonconvex problems, where both the convergence rate of SGD and that of the optimal stochastic algorithm were known [82], [83], [105].”
BackgroundSelf-citationMean Field Analysis of Deep Neural Networks
Sirignano, J. A., & Spiliopoulos, K. (2021) · Mathematics of Operations Research · doi:10.1287/moor.2020.1118 · Cited by 112 · cited 1×
“However, as mentioned in [67], the mathematical limit may not be well-defined.”
BackgroundOverparameterized Nonlinear Learning: Gradient Descent Takes the Shortest Path?Preprint
Oymak, S., & Soltanolkotabi, M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.10004 · Cited by 109 · cited 1×
“…years, in which a number of researchers proved polynomial convergence rates to the global optimal, e.g., [20], [18], [21], [22], [23], [24], [25], [26], [27], [28], [29], [30], [31], [32], [33], [34], [35], [36], [37] and sharper generalization errors, e.g., [20], [23], [21], [38], [39], [40].”
A Generalized Neural Tangent Kernel Analysis for Two-layer Neural NetworksPreprint
Chen, Z., Cao, Y. J., Gu, Q., & Zhang, T. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.04026 · Cited by 103
Self-citationRotskoff, G. M., & Vanden‐Eijnden, E. (2018) · arXiv (Cornell University) · Cited by 100
Dynamics of Deep Neural Networks and Neural Tangent HierarchyPreprint
Huang, J., & Yau, H. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.08156 · Cited by 99 · cited 1×
“, Allen-Zhu and Li [21], Hanin and Nica [76], Bai and Lee [77], and Huang and Yau [78], have tried to model NNs beyond a linear approximation of NTK, typically second-order approximation.”
BackgroundA mean-field limit for certain deep neural networksPreprint
Araújo, D., Oliveira, R. I., & Yukimura, D. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.00193 · Cited by 95 · cited 1×
“[69] and Nguyen and Pham [70] investigated the continuous limit of DNNs under the initialization that all weights were i.”
BackgroundWhat Can ResNet Learn Efficiently, Going Beyond Kernels?
Allen-Zhu, Z., & Li, Y. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.10337 · Cited by 92 · cited 5×
“For example, [21], [77] and [79] proposed a training procedure with randomization techniques to extract the second-order approximation, sharpening complexity bounds.”
BackgroundMethodInfluentialBeyond Linearization: On Quadratic and Higher-Order Approximation of Wide Neural NetworksPreprint
Bai, Y., & Lee, J. D. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.01619 · Cited by 88 · cited 3×
“Specifically, [77] showed that complexity bounds for learning polynomials on uniform distributions are lower than those of NTK by a factor of O(d).”
BackgroundWandering Random Measures in the Fleming-Viot Model
Dawson, D. A., & Hochberg, K. J. (1982) · The Annals of Probability · doi:10.1214/aop/1176993767 · Cited by 76 · cited 1×
“One of the attempted approaches is to model DNNs with nested measures (also known as multi-layer measures [72], [73]).”
Neural Temporal Difference and Q Learning Provably Converge to Global Optima
Cai, Q., Yang, Z., Lee, J. D., & Wang, Z. (2023) · Mathematics of Operations Research · doi:10.1287/moor.2023.1370 · Cited by 70
Regularization Matters: Generalization and Optimization of Neural Nets v.s. their Induced Kernel
Wei, C., Lee, J. D., Liu, Q., & Ma, T. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.05369 · Cited by 68 · cited 4×
“Moreover, in [52] a simple d -dimensional distribution was constructed, for which MF needs O ( d ) samples to learn.”
BackgroundMethodInfluentialBackward Feature Correction: How Deep Learning Performs Deep LearningPreprint
Allen-Zhu, Z., & Li, Y. (2020) · arXiv (Cornell University) · Cited by 65
SGD Learns the Conjugate Kernel Class of the Network
Daniely, A. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1702.08503 · Cited by 64 · cited 1×
“One interesting extension of the RF theory that can be used to analyze the behavior of NN is presented in [15] and [16].”
BackgroundTowards Understanding Hierarchical Learning: Benefits of Neural RepresentationsPreprint
Chen, M., Bai, Y., Lee, J. D., Zhao, T., Wang, H., Xiong, C., & Socher, R. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.13436 · Cited by 62 · cited 1×
“For example, [21], [77] and [79] proposed a training procedure with randomization techniques to extract the second-order approximation, sharpening complexity bounds.”
BackgroundJi, Z., & Telgarsky, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.12292 · Cited by 59 · cited 1×
“…years, in which a number of researchers proved polynomial convergence rates to the global optimal, e.g., [20], [18], [21], [22], [23], [24], [25], [26], [27], [28], [29], [30], [31], [32], [33], [34], [35], [36], [37] and sharper generalization errors, e.g., [20], [23], [21], [38], [39], [40].”
Critical Points of Linear Neural Networks: Analytical Forms and Landscape PropertiesPreprint
Zhou, Y., & Liang, Y. (2018) · arXiv (Cornell University) · Cited by 57
Huang, K., Wang, Y., Tao, M., & Zhao, T. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.06262 · Cited by 57
Provably Efficient Neural Estimation of Structural Equation Model: An Adversarial ApproachPreprint
Liao, L., Chen, Y.-L., Yang, Z., Dai, B., Wang, Z., & Kolar, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.01290 · Cited by 51 · cited 1×
“…years, in which a number of researchers proved polynomial convergence rates to the global optimal, e.g., [20], [18], [21], [22], [23], [24], [25], [26], [27], [28], [29], [30], [31], [32], [33], [34], [35], [36], [37] and sharper generalization errors, e.g., [20], [23], [21], [38], [39], [40].”
On Learning Over-parameterized Neural Networks: A Functional Approximation PerspectivePreprint
Su, L., & Yang, P. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.10826 · Cited by 48 · cited 1×
“…years, in which a number of researchers proved polynomial convergence rates to the global optimal, e.g., [20], [18], [21], [22], [23], [24], [25], [26], [27], [28], [29], [30], [31], [32], [33], [34], [35], [36], [37] and sharper generalization errors, e.g., [20], [23], [21], [38], [39], [40].”
Critical Points of Neural Networks: Analytical Forms and Landscape PropertiesPreprint
Zhou, Y., & Liang, Y. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1710.11205 · Cited by 47 · cited 1×
“With the help of restrictive assumptions, or for specialized models, a number of earlier works, e.g., [96], [97], [98], [99], [100], [101], [102], [103], studied the theoretical characterizations of NN landscapes.”
BackgroundDou, X., & Liang, T. (2020) · Journal of the American Statistical Association · doi:10.1080/01621459.2020.1745812 · Cited by 45 · cited 1×
“In this section, we introduce another line of research, which applies the mathematical tools of the meanfield analysis from statistical physics to study two-layer neural networks [46], [47], [48], [49], [50], [51], [52], [51], [53], [54].”
MethodThe Deep Weight PriorPreprint
Atanov, A., Ashukha, A., Struminsky, K., Vetrov, D., & Welling, M. (2018) · UvA-DARE (University of Amsterdam) · doi:10.48550/arxiv.1810.06943 · Cited by 37 · cited 2×
“This was illustrated in [58] and [59], which we present here as well.”
BackgroundConvex Formulation of Overparameterized Deep Neural Networks
Fang, C., Gu, Y., Zhang, W., & Zhang, T. (2022) · IEEE Transactions on Information Theory · doi:10.1109/tit.2022.3163341 · Cited by 37
Self-citationThe Local Elasticity of Neural NetworksPreprint
He, H., & Su, W. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.06943 · Cited by 36 · cited 1×
“In another work, He and Su [112] observed the so-called local elasticity phenomenon where the prediction of a datum x′ will not be significantly affected after an SGD update at a datum x, which is not close to x′.”
BackgroundMean-Field Langevin Dynamics and Energy Landscape of Neural NetworksPreprint
Hu, K., Ren, Z., Šiška, D., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.07769 · Cited by 30 · cited 1×
“In this section, we introduce another line of research, which applies the mathematical tools of the mean-field analysis from statistical physics to study two-layer neural networks [46], [47], [48], [49], [50], [51], [52], [51], [53], [54].”
MethodModeling from Features: a Mean-field Framework for Over-parameterized Deep Neural NetworksPreprint
Fang, C., Lee, J. D., Yang, P., & Zhang, T. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.01452 · Cited by 29
Self-citationTzen, B., & Raginsky, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.01987 · Cited by 28 · cited 1×
“, [20], [18], [21], [22], [23], [24], [25], [26], [27], [28], [29], [30], [31], [32], [33], [34], [35], [36], [37] and sharper generalization errors, e.”
Dou, X., & Liang, T. (2020) · Figshare · doi:10.6084/m9.figshare.12184887 · Cited by 25
Multilevel mutation-selection systems and set-valued duals
Dawson, D. A. (2017) · Journal of Mathematical Biology · doi:10.1007/s00285-017-1145-2 · Cited by 22 · cited 1×
“One of the attempted approaches is to model DNNs with nested measures (also known as multilayer measures [72], [73]).”
Over Parameterized Two-level Neural Networks Can Learn Near Optimal Feature RepresentationsPreprint
Fang, C., Dong, H., & Zhang, T. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.11508 · Cited by 21 · cited 10×
“[58] that a theoretical understanding of feature learning is the key to explain the success of NN.”
BackgroundMethodResultInfluentialSelf-citationHow to Characterize The Landscape of Overparameterized Convolutional Neural Networks
Gu, Y., Zhang, W., Fang, C., Lee, J. D., & Zhang, T. (2020) · Rare & Special e-Zone (The Hong Kong University of Science and Technology) · Cited by 16
Self-citationCan Temporal-Difference and Q-Learning Learn Representation? A Mean-Field TheoryPreprint
Zhang, Y., Cai, Q., Yang, Z., Chen, Y., & Wang, Z. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.04761 · Cited by 13 · cited 1×
“In this section, we introduce another line of research, which applies the mathematical tools of the mean-field analysis from statistical physics to study two-layer neural networks [46], [47], [48], [49], [50], [51], [52], [51], [53], [54].”
MethodTzen, B., & Raginsky, M. (2020) · arXiv (Cornell University) · Cited by 12
A Rigorous Framework for the Mean Field Limit of Multilayer Neural NetworksPreprint
Nguyen, P.-M., & Pham, H. T. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.11443 · Cited by 11 · cited 3×
“There were a number of recent works that attempted to generalize MF [67], [68], [69], [70], [57], [71], [66].”
BackgroundResultZhang, Y., Cai, Q., Yang, Z., & Wang, Z. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.03709 · Cited by 9 · cited 1×
“…years, in which a number of researchers proved polynomial convergence rates to the global optimal, e.g., [20], [18], [21], [22], [23], [24], [25], [26], [27], [28], [29], [30], [31], [32], [33], [34], [35], [36], [37] and sharper generalization errors, e.g., [20], [23], [21], [38], [39], [40].”
Lower Bounds for Non-Convex Stochastic OptimizationPreprint
Arjevani, Y., Carmon, Y., Duchi, J. C., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.02365 · Cited by 6 · cited 1×
“This question was resolved satisfactorily for general nonconvex problems, where both the convergence rate of SGD and that of the optimal stochastic algorithm were known [82], [83], [105].”
BackgroundConvex Formulation of Overparameterized Deep Neural NetworksPreprint
Fang, C., Gu, Y., Zhang, W., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.07626 · Cited by 3 · cited 1×
“Another remarkable work is [68] and the closely related study [75], in which the authors introduced a new technique, called neural feature repopulation (NFR), to reparameterize the DNNs.”
BackgroundSelf-citationLi, Y., Wei, C., & Ma, T. (2019) · arXiv · doi:10.48550/arxiv.1907.04595 · cited 1×
“A few other works have also mentioned the phenomenon that a large initial learning rate leads to better NN solution [43], [44], which cannot be explained by the NTK view very well.”
Background
Research works printed but not linked: 3
- J. Sirignano and K. Spiliopoulos, "Mean field analysis of neural networks: A law of large numbers," SIAM J. Appl. Math., vol. 80, no. 2, pp. 725-752, Jan. 2020. · DOI
- J. Sirignano and K. Spiliopoulos, "Mean field analysis of neural networks: A central limit theorem," Stochastic Processes Their Appl., vol. 130, no. 3, pp. 1820-1852, Mar. 2020.
- D. Erhan, Y. Bengio, A. Courville, and P . Vincent, "Visualizing higher-layer features of a deep network," Univ. Montreal, vol. 1341, no. 3, p. 1, 2009.
Cited by
Edge artificial intelligence for 6g: vision, enabling technologies, and applications
Letaief, K. B., Shi, Y., Lu, J., & Lu, J. (2022) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2021.3126076 · Cited by 829
“empirical risk minimization for training the models is usually nonconvex, which poses significant challenges to guarantee global optimality for the learning algorithms and models [180].”
BackgroundExploring deep neural networks via layer-peeled model: Minority collapse in imbalanced training
Fang, C., He, H., Long, Q., & Su, W. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2103091118 · Cited by 269
Self-citationA Layer-Wise Theoretical Framework for Deep Learning of Convolutional Neural Networks
Nguyen, H.-T., Li, S., & Cheah, C. C. (2022) · IEEE Access · doi:10.1109/access.2022.3147869 · Cited by 94
“For networks with standard structures, most studies have considered over-parameterization networks [17], [18] where the widths of the hidden layers are assumed to be very large so that the problem can be analyzed mathematically.”
BackgroundBayesian Invariant Risk Minimization
Yong, L., Dong, H., Wang, H., & Zhang, T. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01555 · Cited by 106
“Suppose the prediction model consists of a feature extractor and a classifier [16, 17].”
MethodSelf-citationMinimum Excess Risk in Bayesian Learning
Xu, A., & Raginsky, M. (2022) · IEEE Transactions on Information Theory · doi:10.1109/tit.2022.3176056 · Cited by 52
“, when g : X×W → Y can be represented by over-parametrized neural networks [60].”
Background
Metadata
- Authors
- Fang, C., Dong, H., & Zhang, T.
- Affiliations
- University of Pennsylvania (US)
- Hong Kong University of Science and Technology (HK)
- Venue
- Proceedings of the IEEE
- Published
- 2021-05
- Volume
- 109
- Issue
- 5
- Pages
- 683-703
- ISSN
- 0018-9219 (ISSN-L)
- 1558-2256
- OpenAlex
- W3118114660
- Cited by
- 26
- References
- 134
- Citations by year
- 2026: 2 · 2025: 4 · 2024: 5 · 2023: 6 · 2022: 4 · 2021: 5
- FWCI
- 1.77
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- Proceedings of the IEEE (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- Rare & Special e-Zone (The Hong Kong University of Science and Technology) (submittedVersion)
- arXiv (Cornell University)
- Funders
- GRF (16201320)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- overparameterized neural networks, two-layer neural networks, neural tangent kernel, convexity, generalization, learning dynamics, deep learning theory
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.