Approximation by superpositions of a sigmoidal function
Open accessThe DOI matches its Crossref record; 24 of the 25 references it cites were found in a registry or an index.
Abstract
In this paper we demonstrate that finite linear combinations of compositions of a fixed, univariate function and a set of affine functionals can uniformly approximate any continuous function ofn real variables with support in the unit hypercube; only mild conditions are imposed on the univariate function. Our results settle an open question about representability in the class of single hidden layer neural networks. In particular, we show that arbitrary decision regions can be arbitrarily well approximated by continuous feedforward neural networks with only a single internal, hidden layer and any continuous sigmoidal nonlinearity. The paper discusses approximation properties of other possible types of nonlinearities that might be implemented by artificial neural networks.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The ANNs are considered highly flexible, powerful, and scalable models since they provide the possibility to handle large amounts of parameters and demonstrably approximate any function (Cybenko 1989; Gerón 2019).”
From Software sensors in the monitoring of microalgae cultivations (2024) · cites this paper as Cybenko 1989
In the citing paper
“These theorems guarantee that, provided enough free parameters, feedforward neural networks can approximate any continuous function on a closed and bounded subset of R n to an arbitrary degree of accuracy.”
From Continuous-variable quantum neural networks (2019) · cites this paper as [55]
In the citing paper
“For ordinal deep learning models, such as FFNs and CNNs, deep architecture immediately benefits from the universal approximation power and expressive capacity (Cybenko, 1989; Hornik, 1991; Telgarsky, 2016; Lu et al., 2017; Petersen & Voigtlaender, 2020; Zhou, 2020).”
From Anti-Oversmoothing in Deep Vision Transformers via the Fourier Domain Analysis: From Theory to Practice (2022) · cites this paper as Cybenko, 1989
In the citing paper
“An artificial neural network model composed of more than two layers is known to be able to approximate any continuous mapping (Cybenko, 1989; Funahashi, 1989; Hornik et al., 1989), which is a theoretical basis of the universal cerebellar transform (Schmahmann, 2004).”
From The Cerebro-Cerebellum as a Locus of Forward Model: A Review (2020) · cites this paper as Cybenko, 1989
In the citing paper
“For example, in a generic M-theory setting (without localized sources), I classical is given by I classical = I F 4 = 1 6 g 11-d (e (d- 8)A (12 -d) f 2 4 -d e dA f 2 4 ) ≤ 0 , (D.15) where we are working with F 4 = F 7 and we have decomposed F 4 = f4 + f 4 , with f4 being the space-time component (which is non-vanishing only for d = 4).The last inequality follows from f 2 4 ≤ 0 23 .”
From Hyperbolic compactification of M-theory and de Sitter quantum gravity (2022) · cites this paper as [66]
5 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Multilayer feedforward networks are universal approximators
Hornik, K., Stinchcombe, M., & White, H. (1989) · Neural Networks · doi:10.1016/0893-6080(89)90020-8 · Cited by 21,728
Parallel Distributed Processing: Explorations in the Microstructure of Cognition: Foundations
Rumelhart, D. E., McClelland, J. L., & AU (1986) · doi:10.7551/mitpress/5236.001.0001 · Cited by 15,686
Rudin, W. (1973) · Cited by 13,740
Garreau, G. A., & Rudin, W. (1987) · The Statistician · doi:10.2307/2348852 · Cited by 10,622
An introduction to computing with neural nets
Lippmann, R. (1987) · IEEE ASSP Magazine · doi:10.1109/massp.1987.1165576 · Cited by 7,825
On the approximate realization of continuous mappings by neural networks
Funahashi, K.-I. (1989) · Neural Networks · doi:10.1016/0893-6080(89)90003-8 · Cited by 4,206
Valiant, L. G. (1984) · Communications of the ACM · doi:10.1145/1968.1972 · Cited by 3,279
What Size Net Gives Valid Generalization?
Baum, E. B., & Haussler, D. (1989) · Neural Computation · doi:10.1162/neco.1989.1.1.151 · Cited by 1,541
Huber, P. J. (1985) · The Annals of Statistics · doi:10.1214/aos/1176349519 · Cited by 1,277
Real Analysis and Probability.
Ross, K. A., & Ash, R. B. (1975) · The American Mathematical Monthly · doi:10.2307/2319153 · Cited by 714
Research works printed but not linked: 5
- [K] A. N. Kolmogorov, On the representation of continuous functions of many variables by superposition of continuous functions of one variable and addition,Dokl. Akad. Nauk. SSSR,114 (1957), 953–956.
- [MSJ] J. Makhoul, R. Schwartz, and A. El-Jaroudi, Classification capabilities of two-layer neural nets.Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing, Glasgow, 1989 (to appear).
- [MP] M. Minsky and S. Papert,Perceptrons, MIT Press, Cambridge, MA, 1969.
- [N] N. J. Nilsson,Learning Machines, McGraw-Hill, New York, 1965.
- [WL] A Wieland and R. Leighton, Geometric analysis of neural network capabilities,Proceedings of IEEE First International Conference on Neural Networks, San Diego, CA, pp. III-385–III-392, 1987.
Cited by
Items 1,001–1,050 of 1,050
Bandai, T., & Ghezzehei, T. A. (2022) · Hydrology and Earth System Sciences · doi:10.5194/hess-26-4469-2022 · Cited by 111
Modeling of CO2 capture ability of [Bmim][BF4] ionic liquid using connectionist smart paradigms
Daryayehsalameh, B., Nabavi, M., & Vaferi, B. (2021) · Environmental Technology & Innovation · doi:10.1016/j.eti.2021.101484 · Cited by 111
Zhou, H. F., Ni, Y. Q., & Ko, J. M. (2010) · Engineering Structures · doi:10.1016/j.engstruct.2010.02.026 · Cited by 102
Artificial Neural Networks (ANNs) as a Novel Modeling Technique in Tribology
Argatov, I. (2019) · Frontiers in Mechanical Engineering · doi:10.3389/fmech.2019.00030 · Cited by 97
“This nonlinear regression problem was considered by Cybenko (1989), who proved a theorem stating that any continuous function f (x) defined on the n-dimensional unit hypercube [0, 1]n can be uniformly approximated by a sum F(x) = N ∑ j=1 αjσ ( wj · x+ bj ) , (1) where wj · x = wj1x1 + wj2x2 + . . .…”
BackgroundDeep Nets: What have They Ever Done for Vision?
Yuille, A. L., & Liu, C. (2021) · International Journal of Computer Vision · doi:10.1007/s11263-020-01405-z · Cited by 98
“Theoretical results show that multi-layer perceptrons, and hence Deep Nets, can represent any input output function provided there are a sufficient number of hidden units (Cybenko, 1989; Hornik et al., 1989).”
BackgroundA predictive-control framework to address bus bunching
Andres, M., & Nair, R. (2017) · Transportation Research Part B: Methodological · doi:10.1016/j.trb.2017.06.013 · Cited by 95
Statistical Foundations of Actuarial Learning and its Applications
Wüthrich, M. V., & Merz, M. (2023) · Springer Actuarial · doi:10.1007/978-3-031-12409-9 · Cited by 95
On the Universality of the Quantum Approximate Optimization Algorithm
Morales, M. E. S., Biamonte, J., & Zimborás, Z. (2019) · arXiv · doi:10.48550/arxiv.1909.03123 · Cited by 95
Towards Real-Time Object Detection on Embedded Systems
Mao, H., Yao, S., Tang, T., Li, B., Yao, J., & Wang, Y. (2018) · IEEE Transactions on Emerging Topics in Computing · doi:10.1109/tetc.2016.2593643 · Cited by 94
Nokkala, J., Martínez-Peña, R., Giorgi, G. L., Parigi, V., Soriano, M. C., & Zambrini, R. (2020) · arXiv · doi:10.48550/arxiv.2006.04821 · Cited by 94
Wireless Network Intelligence at the Edge
Park, J., Samarakoon, S., Bennis, M., & Debbah, M. (2019) · Proceedings of the IEEE · doi:10.1109/jproc.2019.2941458 · Cited by 92
“The proof is provided for shallow NNs [83], [84] and recently for deep NNs [85].”
BackgroundShaping dynamics with multiple populations in low-rank recurrent networks
Beiran, M., Dubreuil, A., Valente, A., Mastrogiuseppe, F., & Ostojic, S. (2021) · Neural Computation · doi:10.1162/neco_a_01381 · Cited by 92
“…the universality property has been shown using a wide range of transfer functions: squashing or sigmoidal functions (Funahashi, 1989; Hornik et al., 1989; Cybenko, 1989), Heaviside functions, sinusoidal functions (Gallant & White, 1988), and radial basis functions (Park & Sandberg, 1991).”
BackgroundInfluentialDetection of Cyber-attacks to indoor real time localization systems for autonomous robots
Guerrero-Higueras, Á. M., DeCastro-García, N., & Matellán, V. (2018) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2017.10.006 · Cited by 91
Eliminating Temperature Effect in Vibration-Based Structural Damage Detection
Zhou, H. F., Ni, Y. Q., & Ko, J. M. (2011) · Journal of Engineering Mechanics · doi:10.1061/(asce)em.1943-7889.0000273 · Cited by 91
Kelly, B., & Xiu, D. (2023) · Foundations and Trends® in Finance · doi:10.1561/0500000064 · Cited by 89
A Neural Network-Based On-device Learning Anomaly Detector for Edge Devices
Tsukada, M., Kondo, M., & Matsutani, H. (2020) · IEEE Transactions on Computers · doi:10.1109/tc.2020.2973631 · Cited by 81
“Ghidden {Sigmoid [32], Identity (4)} {Sigmoid, Identity} pðxÞ Uniform [0,1] Uniform [0,1] L MSE(5) MSE ai {0.”
BackgroundTraffic prediction using neural networks
Yu, E. S., & Chen, C. Y. R. (2002) · Proceedings of GLOBECOM '93. IEEE Global Telecommunications Conference · doi:10.1109/glocom.1993.318226 · Cited by 77
Language Model Crossover: Variation through Few-Shot Prompting
Meyerson, E., Nelson, M. J., Bradley, H., Gaier, A., Moradi, A., Hoover, A. K., & Lehman, J. (2024) · ACM Transactions on Evolutionary Learning and Optimization · doi:10.1145/3694791 · Cited by 75
“These results are suggestive of the versatility of this biasing technique; a variety of sorting strategies could be employed to cater to specific objectives.”
Language model crossover: Variation through few-shot prompting
Meyerson, E., Nelson, M. J., Bradley, H., Gaier, A., Moradi, A., Hoover, A. K., & Lehman, J. (2023) · arXiv · doi:10.48550/arxiv.2302.12170 · Cited by 75
Forecasting the Suez Canal traffic: A neural network analysis
Mostafa, M. M. (2004) · Maritime Policy & Management · doi:10.1080/0308883032000174463 · Cited by 72
“Based on the proof in Cybenko [60] and Hornik et al .”
MethodTowards Revealing the Mystery behind Chain of Thought: a Theoretical Perspective
Feng, G., Zhang, B., Gu, Y., Ye, H., He, D., & Wang, L. (2023) · arXiv · doi:10.48550/arxiv.2305.15408 · Cited by 69
“, the depth or width approaches infinity) are already universal function approximators [15, 30, 34].”
BackgroundMultikernel Capsule Network for Schizophrenia Identification
Wang, T., Bezerianos, A., Cichocki, A., & Li, J. (2022) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2020.3035282 · Cited by 63
Divate, M., Tyagi, A., Richard, D. J., Prasad, P. A., Gowda, H., & Nagaraj, S. H. (2022) · Cancers · doi:10.3390/cancers14051185 · Cited by 55
“According to the universal approximation theorem [5,6], a feedforward network with linear output and at least one hidden layer with any squashing activation function can approximate any decision boundary to arbitrary closeness if provided with sufficient hidden units.”
BackgroundFuzzy adaptive decision-making for boundedly rational traders in speculative stock markets
Bekiros, S. D. (2010) · European Journal of Operational Research · doi:10.1016/j.ejor.2009.04.015 · Cited by 53
An End to Endless Forms: Epistasis, Phenotype Distribution Bias, and Nonuniform Evolution
Borenstein, E., & Krakauer, D. C. (2008) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1000202 · Cited by 55
“function approximator), as proven for the Cybenko theorem [22], able to achieve linear separability of inputs as is required by, for example, the XOR function.”
BackgroundBasic Framework and Main Methods of Uncertainty Quantification
Zhang, J., Yin, J., & Wang, R. (2020) · Mathematical Problems in Engineering · doi:10.1155/2020/6068203 · Cited by 52
Predicting Wireless Channel Features Using Neural Networks
Navabi, S., Wang, C., Bursalioglu, O. Y., & Papadopoulos, H. (2018) · 2018 IEEE International Conference on Communications (ICC) · doi:10.1109/icc.2018.8422221 · Cited by 51
NNWarp: Neural Network-based Nonlinear Deformation
Luo, R., Shao, T., Wang, H., Xu, W., Chen, X., Zhou, K., & Yang, Y. (2018) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2018.2881451 · Cited by 50
“The complexity of a neural network highly depends on its input [9].”
BackgroundSoftware sensors in the monitoring of microalgae cultivations
Porras Reyes, L., Havlik, I., & Beutel, S. (2024) · Reviews in Environmental Science and Bio/Technology · doi:10.1007/s11157-023-09679-8 · Cited by 48
Deep learning in spatial transcriptomics: Learning from the next next-generation sequencing
Heydari, A. A., & Sindi, S. S. (2023) · Biophysics Reviews · doi:10.1063/5.0091135 · Cited by 44
Deep Learning in Spatial Transcriptomics: Learning From the Next Next-Generation Sequencing
Heydari, A. A., & Sindi, S. S. (2022) · doi:10.1101/2022.02.28.482392 · Cited by 44
Information-Theoretic Probing for Linguistic Structure
Pimentel, T., Valvoda, J., Hall Maudslay, R., Zmigrod, R., Williams, A., & Cotterell, R. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.420 · Cited by 41
Hyperbolic compactification of M-theory and de Sitter quantum gravity
De Luca, G. B., Silverstein, E., & Torroba, G. (2022) · SciPost Physics · doi:10.21468/scipostphys.12.3.083 · Cited by 39
“The Universal Approximation Theorem guarantees that (8.5) can approximate arbitrarily well a large class of functions.15 The functional form in (8.5) is the simplest architecture, but many others have been developed with features adapted to the problem at hand or that enjoy some particular properties.”
MethodInfluentialDomingos, L. C. F., Santos, P. E., Skelton, P. S. M., Brinkworth, R. S. A., & Sammut, K. (2022) · IEEE Access · doi:10.1109/access.2022.3220265 · Cited by 51
ANN Prediction of Metabolic Syndrome: a Complex Puzzle that will be Completed
Ivanović, D., Kupusinac, A., Stokić, E., Doroslovački, R., & Ivetić, D. (2016) · Journal of Medical Systems · doi:10.1007/s10916-016-0601-7 · Cited by 31
Uncertainty Quantification in Deep Learning for Safer Neuroimage Enhancement
Tanno, R., Worrall, D., Kaden, E., Ghosh, A., Grussu, F., Bizzi, A., Sotiropoulos, S. N., Criminisi, A., & Alexander, D. C. (2019) · arXiv · doi:10.48550/arxiv.1907.13418 · Cited by 30
Cadini, F., Lomazzi, L., Ferrater Roca, M., Sbarufatti, C., & Giglio, M. (2022) · Mechanical Systems and Signal Processing · doi:10.1016/j.ymssp.2021.108048 · Cited by 25
On the analysis of the inverse problem of metabolic pathways using artificial neural networks
Mendes, P., & Kell, D. B. (1996) · Biosystems · doi:10.1016/0303-2647(95)01565-5 · Cited by 24
A Connectome Based Hexagonal Lattice Convolutional Network Model of the Drosophila Visual System
Tschopp, F. D., Reiser, M. B., & Turaga, S. C. (2018) · arXiv · doi:10.48550/arxiv.1806.04793 · Cited by 24
Eckstein, M. K., Summerfield, C., Daw, N. D., & Miller, K. J. (2023) · doi:10.1101/2023.05.17.541226 · Cited by 18
Dan, A., Liu, B., Patil, U., Manuraj, B. N. M., Gandhi, R., Buchel, J., Chundawat, S. P. S., Guo, W., & Ramachandran, R. (2025) · Control Engineering Practice · doi:10.1016/j.conengprac.2024.106198 · Cited by 20
Self-supervised learning with physics-aware neural networks – I. Galaxy model fitting
Aragon-Calvo, M. A., & Carvajal, J. C. (2020) · Monthly Notices of the Royal Astronomical Society · doi:10.1093/mnras/staa2228 · Cited by 17
Acceleration of Chemical Kinetics Computation with the Learned Intelligent Tabulation (LIT) Method
Haghshenas, M., Mitra, P., Santo, N. D., & Schmidt, D. P. (2021) · Energies · doi:10.3390/en14237851 · Cited by 14
Lewis, T. G., & Denning, P. J. (2018) · Communications of the ACM · doi:10.1145/3286868 · Cited by 14
Adaptive control of reaction–diffusion PDEs via neural operator-approximated gain kernels
Bhan, L., Shi, Y., & Krstic, M. (2025) · Systems & Control Letters · doi:10.1016/j.sysconle.2024.105968 · Cited by 12
Utgés, J. S., MacGowan, S. A., Ives, C. M., & Barton, G. J. (2024) · Communications Biology · doi:10.1038/s42003-024-05970-8 · Cited by 10
Macro, Finance, and Macro Finance: Solving Nonlinear Models in Continuous Time with Machine Learning
Duarte, V. (2017) · SSRN Electronic Journal · doi:10.2139/ssrn.3012602 · Cited by 9
Multimodal Data Fusion based on the Global Workspace Theory
Bao, C., Fountas, Z., Olugbade, T., & Bianchi-Berthouze, N. (2020) · arXiv · doi:10.48550/arxiv.2001.09485 · Cited by 9
Time-dependent Decentralized Routing using Federated Learning
Wilbur, M., Samal, C., Talusan, J. P., Yasumoto, K., & Dubey, A. (2020) · 2020 IEEE 23rd International Symposium on Real-Time Distributed Computing (ISORC) · doi:10.1109/isorc49007.2020.00018 · Cited by 8
“For binary classification, we used a sigmoid function [29] for the output layer and an Adam optimizer was used as the optimizer for the model.”
MethodWang, P., Zheng, W., Chen, T., & Wang, Z. (2022) · arXiv · doi:10.48550/arxiv.2203.05962 · Cited by 80
“…deep learning models, such as FFNs and CNNs, deep architecture immediately benefits from the universal approximation power and expressive capacity (Cybenko, 1989; Hornik, 1991; Telgarsky, 2016; Lu et al., 2017; Petersen & Voigtlaender, 2020; Zhou, 2020 (Vaswani et al., 2017; Brown et al., 2020),…”
Background
Metadata
- Authors
- Cybenko, G.
- Affiliations
- University of Illinois Urbana-Champaign (US)
- Corresponding author
- Cybenko, G.
- ORCID
- Cybenko, G.
- Venue
- Mathematics of Control, Signals, and Systems
- Published
- 1989-12
- Volume
- 2
- Issue
- 4
- Pages
- 303-314
- ISSN
- 0932-4194 (ISSN-L)
- 1435-568X
- OpenAlex
- W2103496339
- Semantic Scholar
- 8da1dda34ecc96263102181448c94ec7d645d085
- Cited by
- 13,883
- References
- 25
- Citations by year
- 2026: 640 · 2025: 889 · 2024: 793 · 2023: 746 · 2022: 827 · 2021: 1050 · 2020: 1001 · 2019: 761 · 2018: 542 · 2017: 414 · 2016: 312 · 2015: 313 · 2014: 289 · 2013: 310 · 2012: 296
- FWCI
- 94.52
- Type
- journal article
- Language
- en
- Open access
- green
- hal.science
- Licence
- cc-by-nc
- www.springer.com/tdm (tdm)
- Locations
- Mathematics of Control Signals and Systems (publishedVersion)
- HAL (submittedVersion, cc-by-nc)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and ELM
- Physical Sciences › Mathematics › Applied Mathematics › Approximation Theory and Sequence Spaces
- Keywords
- universal approximation, sigmoidal function, single hidden layer neural network, feedforward neural networks, approximation of continuous functions, decision regions, nonlinear activation functions, unit hypercube
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.