Approximation by superpositions of a sigmoidal function
Open accessThe DOI matches its Crossref record; 24 of the 25 references it cites were found in a registry or an index.
Abstract
In this paper we demonstrate that finite linear combinations of compositions of a fixed, univariate function and a set of affine functionals can uniformly approximate any continuous function ofn real variables with support in the unit hypercube; only mild conditions are imposed on the univariate function. Our results settle an open question about representability in the class of single hidden layer neural networks. In particular, we show that arbitrary decision regions can be arbitrarily well approximated by continuous feedforward neural networks with only a single internal, hidden layer and any continuous sigmoidal nonlinearity. The paper discusses approximation properties of other possible types of nonlinearities that might be implemented by artificial neural networks.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The ANNs are considered highly flexible, powerful, and scalable models since they provide the possibility to handle large amounts of parameters and demonstrably approximate any function (Cybenko 1989; Gerón 2019).”
From Software sensors in the monitoring of microalgae cultivations (2024) · cites this paper as Cybenko 1989
In the citing paper
“These theorems guarantee that, provided enough free parameters, feedforward neural networks can approximate any continuous function on a closed and bounded subset of R n to an arbitrary degree of accuracy.”
From Continuous-variable quantum neural networks (2019) · cites this paper as [55]
In the citing paper
“For ordinal deep learning models, such as FFNs and CNNs, deep architecture immediately benefits from the universal approximation power and expressive capacity (Cybenko, 1989; Hornik, 1991; Telgarsky, 2016; Lu et al., 2017; Petersen & Voigtlaender, 2020; Zhou, 2020).”
From Anti-Oversmoothing in Deep Vision Transformers via the Fourier Domain Analysis: From Theory to Practice (2022) · cites this paper as Cybenko, 1989
In the citing paper
“An artificial neural network model composed of more than two layers is known to be able to approximate any continuous mapping (Cybenko, 1989; Funahashi, 1989; Hornik et al., 1989), which is a theoretical basis of the universal cerebellar transform (Schmahmann, 2004).”
From The Cerebro-Cerebellum as a Locus of Forward Model: A Review (2020) · cites this paper as Cybenko, 1989
In the citing paper
“For example, in a generic M-theory setting (without localized sources), I classical is given by I classical = I F 4 = 1 6 g 11-d (e (d- 8)A (12 -d) f 2 4 -d e dA f 2 4 ) ≤ 0 , (D.15) where we are working with F 4 = F 7 and we have decomposed F 4 = f4 + f 4 , with f4 being the space-time component (which is non-vanishing only for d = 4).The last inequality follows from f 2 4 ≤ 0 23 .”
From Hyperbolic compactification of M-theory and de Sitter quantum gravity (2022) · cites this paper as [66]
5 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Multilayer feedforward networks are universal approximators
Hornik, K., Stinchcombe, M., & White, H. (1989) · Neural Networks · doi:10.1016/0893-6080(89)90020-8 · Cited by 21,728
Parallel Distributed Processing: Explorations in the Microstructure of Cognition: Foundations
Rumelhart, D. E., McClelland, J. L., & AU (1986) · doi:10.7551/mitpress/5236.001.0001 · Cited by 15,686
Rudin, W. (1973) · Cited by 13,740
Garreau, G. A., & Rudin, W. (1987) · The Statistician · doi:10.2307/2348852 · Cited by 10,622
An introduction to computing with neural nets
Lippmann, R. (1987) · IEEE ASSP Magazine · doi:10.1109/massp.1987.1165576 · Cited by 7,825
On the approximate realization of continuous mappings by neural networks
Funahashi, K.-I. (1989) · Neural Networks · doi:10.1016/0893-6080(89)90003-8 · Cited by 4,206
Valiant, L. G. (1984) · Communications of the ACM · doi:10.1145/1968.1972 · Cited by 3,279
What Size Net Gives Valid Generalization?
Baum, E. B., & Haussler, D. (1989) · Neural Computation · doi:10.1162/neco.1989.1.1.151 · Cited by 1,541
Huber, P. J. (1985) · The Annals of Statistics · doi:10.1214/aos/1176349519 · Cited by 1,277
Real Analysis and Probability.
Ross, K. A., & Ash, R. B. (1975) · The American Mathematical Monthly · doi:10.2307/2319153 · Cited by 714
A general framework for parallel distributed processing
Rumelhart, D. E., Hinton, G. E., & McClelland, J. L. (1986) · The MIT Press eBooks · Cited by 711
Griffin, D. W., & Lim, J. S. (1988) · IEEE Transactions on Acoustics, Speech, and Signal Processing · doi:10.1109/29.1651 · Cited by 494
(1972) · Elsevier eBooks · doi:10.1016/c2013-0-07155-1 · Cited by 445
Construction of neural nets using the radon transform
Carroll, & Dickinson (1989) · International Joint Conference on Neural Networks · doi:10.1109/ijcnn.1989.118639 · Cited by 162
Classifying learnable geometric concepts with the Vapnik-Chervonenkis dimension
Blumer, A., Ehrenfeucht, A., Haussler, D., & Warmuth, M. (1986) · Proceedings of the eighteenth annual ACM symposium on Theory of computing - STOC '86 · doi:10.1145/12130.12158 · Cited by 159
On Nonlinear Functions of Linear Combinations
Diaconis, P., & Shahshahani, M. (1984) · SIAM Journal on Scientific and Statistical Computing · doi:10.1137/0905013 · Cited by 149
Constructive approximations for neural networks by sigmoidal functions
Jones, L. K. (1990) · Proceedings of the IEEE · doi:10.1109/5.58342 · Cited by 146
Introduction to neural networks for intelligent control
Bavarian, B. (1988) · IEEE Control Systems Magazine · doi:10.1109/37.1866 · Cited by 145
Spectral synthesis and the Pompeiu problem
Brown, L., Schreiber, B. M., & Taylor, B. A. (1973) · Annales de l’institut Fourier · doi:10.5802/aif.474 · Cited by 139
On representation and approximation of nonlinear systems
Palm, G. (1979) · Biological Cybernetics · doi:10.1007/bf00336857 · Cited by 93
On representation and approximation of nonlinear systems
Palm, G. (1978) · Biological Cybernetics · doi:10.1007/bf00344243 · Cited by 79
Lorentz, G. G. (1976) · Proceedings of symposia in pure mathematics · doi:10.1090/pspum/028.2/0507425 · Cited by 40
Some remarks on ridge functions
Dahmen, W., & Micchelli, C. A. (1987) · RWTH Publications (RWTH Aachen) · Cited by 25
Valiant, L. G. (2026) · Foundations of Computation and Machine Learning · doi:10.1145/3749740.3749746
Research works printed but not linked: 5
- [K] A. N. Kolmogorov, On the representation of continuous functions of many variables by superposition of continuous functions of one variable and addition,Dokl. Akad. Nauk. SSSR,114 (1957), 953–956.
- [MSJ] J. Makhoul, R. Schwartz, and A. El-Jaroudi, Classification capabilities of two-layer neural nets.Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing, Glasgow, 1989 (to appear).
- [MP] M. Minsky and S. Papert,Perceptrons, MIT Press, Cambridge, MA, 1969.
- [N] N. J. Nilsson,Learning Machines, McGraw-Hill, New York, 1965.
- [WL] A Wieland and R. Leighton, Geometric analysis of neural network capabilities,Proceedings of IEEE First International Conference on Neural Networks, San Diego, CA, pp. III-385–III-392, 1987.
Cited by
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
Multilayer feedforward networks are universal approximators
Hornik, K., Stinchcombe, M., & White, H. (1989) · Neural Networks · doi:10.1016/0893-6080(89)90020-8 · Cited by 21,728
Approximation capabilities of multilayer feedforward networks
Hornik, K. (1991) · Neural Networks · doi:10.1016/0893-6080(91)90009-t · Cited by 6,091
Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators
Lu, L., Jin, P., Pang, G., Zhang, Z., & Karniadakis, G. E. (2021) · Nature Machine Intelligence · doi:10.1038/s42256-021-00302-5 · Cited by 3,418
XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks
Rastegari, M., Ordonez, V., Redmon, J., & Farhadi, A. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_32 · Cited by 4,408
Metadata
- Authors
- Cybenko, G.
- Affiliations
- University of Illinois Urbana-Champaign (US)
- Corresponding author
- Cybenko, G.
- ORCID
- Cybenko, G.
- Venue
- Mathematics of Control, Signals, and Systems
- Published
- 1989-12
- Volume
- 2
- Issue
- 4
- Pages
- 303-314
- ISSN
- 0932-4194 (ISSN-L)
- 1435-568X
- OpenAlex
- W2103496339
- Semantic Scholar
- 8da1dda34ecc96263102181448c94ec7d645d085
- Cited by
- 13,883
- References
- 25
- Citations by year
- 2026: 640 · 2025: 889 · 2024: 793 · 2023: 746 · 2022: 827 · 2021: 1050 · 2020: 1001 · 2019: 761 · 2018: 542 · 2017: 414 · 2016: 312 · 2015: 313 · 2014: 289 · 2013: 310 · 2012: 296
- FWCI
- 94.52
- Type
- journal article
- Language
- en
- Open access
- green
- hal.science
- Licence
- cc-by-nc
- www.springer.com/tdm (tdm)
- Locations
- Mathematics of Control Signals and Systems (publishedVersion)
- HAL (submittedVersion, cc-by-nc)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and ELM
- Physical Sciences › Mathematics › Applied Mathematics › Approximation Theory and Sequence Spaces
- Keywords
- universal approximation, sigmoidal function, single hidden layer neural network, feedforward neural networks, approximation of continuous functions, decision regions, nonlinear activation functions, unit hypercube
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.