On the Spectral Bias of Neural Networks
Open accessThe DOI matches its OpenAlex record; 8 of the 8 references it cites were found in a registry or an index.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Multilayer feedforward networks are universal approximators
Hornik, K., Stinchcombe, M., & White, H. (1989) · Neural Networks · doi:10.1016/0893-6080(89)90020-8 · Cited by 21,728 · cited 2×
“While universal approximation properties of neural networks have been known since the early 90s (Hornik et al., 1989; Cybenko, 1989; Leshno et al., 1993), recent research has shed light on the mechanisms underlying such expressivity (Montufar et al., 2014; Raghu et al., 2016; Poole et al., 2016).”
BackgroundResultApproximation by superpositions of a sigmoidal function
Cybenko, G. (1989) · Mathematics of Control, Signals, and Systems · doi:10.1007/bf02551274 · Cited by 13,883
Learning Deep Architectures for AI
Bengio, Y. (2009) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000006 · Cited by 9,239
Self-citationUnderstanding deep learning requires rethinking generalization
Zhang, C., Bengio, S., Hardt, M., Recht, B., & Vinyals, O. (2016) · arXiv · doi:10.48550/arxiv.1611.03530 · Cited by 2,838 · cited 6×
“Poole et al., 2016). It was recently established that DNNs can in fact memorize random data to 100% training accuracy showing that they are indeed highly expressive in the over-parametrized regime2 (Zhang et al., 2017). This has sparked new interest in studying different aspects of deep networks to understand why they…”
BackgroundMethodResultInfluentialA Closer Look at Memorization in Deep Networks
Arpit, D., Jastrzębski, S., Ballas, N., Krueger, D., Bengio, E., Kanwal, M. S., Maharaj, T., Fischer, A., Courville, A., Bengio, Y., et al. (2017) · arXiv · doi:10.48550/arxiv.1706.05394 · Cited by 1,307 · cited 5×
“Even though the intrinsic manifold for images is unknown, our theoretical argument might explain the observation in Arpit et al. (2017), where it is shown that it is easier to fit label (i.e. a high-frequency signal) to CIFAR-10 than it is to random inputs. For the latter, the underlying "latent manifold" is the Eucli…”
BackgroundSelf-citationOn the Number of Linear Regions of Deep Neural Networks
Montúfar, G., Pascanu, R., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1402.1869 · Cited by 840
Self-citationBenefits of Depth in Neural Networks
Telgarsky, M. (2016) · arXiv · doi:10.48550/arxiv.1602.04485 · Cited by 341
Neural network with unbounded activation functions is universal approximator
Sonoda, S., & Murata, N. (2017) · Applied and Computational Harmonic Analysis · doi:10.1016/j.acha.2015.12.005 · Cited by 303 · cited 1×
“This approach has been recently generalized to unbounded activation functions by Sonoda & Murata (2017).”
Background
Cited by
Physics-informed machine learning
Karniadakis, G. E., Kevrekidis, I. G., Lu, L., Perdikaris, P., Wang, S., & Yang, L. (2021) · Nature Reviews Physics · doi:10.1038/s42254-021-00314-5 · Cited by 8,718
“For example, refs 76,77,173 have identified two fundamental weaknesses of PINNs, relating spectral bias 170 to a discrepancy in the convergence rate of different components in a PINN loss function.”
DeepXDE: A Deep Learning Library for Solving Differential Equations
Lu, L., Meng, X., Mao, Z., & Karniadakis, G. E. (2021) · SIAM Review · doi:10.1137/19m1274067 · Cited by 2,512
When and why PINNs fail to train: A neural tangent kernel perspective
Wang, S., Yu, X., & Perdikaris, P. (2022) · Journal of Computational Physics · doi:10.1016/j.jcp.2021.110768 · Cited by 1,496
Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains
Tancik, M., Srinivasan, P. P., Mildenhall, B., Fridovich-Keil, S., Raghavan, N., Singhal, U., Ramamoorthi, R., Barron, J. T., & Ng, R. (2020) · arXiv · doi:10.48550/arxiv.2006.10739 · Cited by 1,459
“Analysis of the NTK’s eigendecomposition shows that its eigenvalue spectrum decays rapidly as a function of frequency, which explains the widely-observed “spectral bias” of deep networks towards learning low-frequency functions [3, 4, 33].”
BackgroundD-NeRF: Neural Radiance Fields for Dynamic Scenes
Pumarola, A., Corona, E., Pons-Moll, G., & Moreno-Noguer, F. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01018 · Cited by 1,355
Metadata
- Authors
- Rahaman, N., Baratin, A., Arpit, D., Draxler, F., Lin, M., Hamprecht, F. A., Bengio, Y., & Courville, A. C.
- Venue
- arXiv (Cornell University)
- Publisher
- Cornell University
- Published
- 2018-06-22
- Pages
- 5301-5310
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1806.08734
- OpenAlex
- W2897097528
- Cited by
- 139
- References
- 8
- Type
- posted content
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: On the Spectral Bias of Neural Networks. https://app.cochat.ai/papers/10.48550/arxiv.1806.08734
- The paper
- Rahaman, N., Baratin, A., Arpit, D., Draxler, F., Lin, M., Hamprecht, F. A., Bengio, Y., & Courville, A. C. (2018). On the Spectral Bias of Neural Networks. arXiv (Cornell University). https://doi.org/10.48550/arxiv.1806.08734
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.