The DOI matches its Crossref record; 108 of the 108 references it cites were found in a registry or an index.
Constructive algorithms for structure learning in feedforward neural networks for regression problems
Abstract
In this survey paper, we review the constructive algorithms for structure learning in feedforward neural networks for regression problems. The basic idea is to start with a small network, then add hidden units and weights incrementally until a satisfactory solution is found. By formulating the whole problem as a state-space search, we first describe the general issues in constructive algorithms, with special emphasis on the search strategy. A taxonomy, based on the differences in the state transition mapping, the training algorithm, and the network architecture, is then presented.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A new look at the statistical model identification
Akaike, H. (1974) · IEEE Transactions on Automatic Control · doi:10.1109/tac.1974.1100705 · Cited by 51,879
Estimating the Dimension of a Model
Schwarz, G. (1978) · The Annals of Statistics · doi:10.1214/aos/1176344136 · Cited by 47,033
An Introduction to the Bootstrap
Efron, B., & Tibshirani, R. J. (1994) · doi:10.1201/9780429246593 · Cited by 40,500
An Introduction to the Bootstrap
Efron, B., & Tibshirani, R. J. (1993) · doi:10.1007/978-1-4899-4541-9 · Cited by 21,230
Approximation by superpositions of a sigmoidal function
Cybenko, G. (1989) · Mathematics of Control, Signals, and Systems · doi:10.1007/bf02551274 · Cited by 13,883
(1973) · CERN Document Server (European Organization for Nuclear Research) · Cited by 12,895
Cross validatory choice and assessment of statistical predictions
Stone, M. (1974) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1974.tb00994.x · Cited by 10,881
Approximation capabilities of multilayer feedforward networks
Hornik, K. (1991) · Neural Networks · doi:10.1016/0893-6080(91)90009-t · Cited by 6,091
Modeling by shortest data description
Rissanen, J. (1978) · Automatica · doi:10.1016/0005-1098(78)90005-5 · Cited by 6,011
MacKay, D. J. C. (1992) · Neural Computation · doi:10.1162/neco.1992.4.3.415 · Cited by 4,388
Bayesian Learning for Neural Networks
Neal, R. M. (1996) · Lecture Notes in Statistics · doi:10.1007/978-1-4612-0745-0 · Cited by 4,276
On the approximate realization of continuous mappings by neural networks
Funahashi, K.-I. (1989) · Neural Networks · doi:10.1016/0893-6080(89)90003-8 · Cited by 4,206
Universal Approximation Using Radial-Basis-Function Networks
Park, J., & Sandberg, I. W. (1991) · Neural Computation · doi:10.1162/neco.1991.3.2.246 · Cited by 4,112
Generalized Cross-Validation as a Method for Choosing a Good Ridge Parameter
Golub, G. H., Heath, M., & Wahba, G. (1979) · Technometrics · doi:10.1080/00401706.1979.10489751 · Cited by 3,934
Neural Networks and the Bias/Variance Dilemma
Geman, S., Bienenstock, E., & Doursat, R. (1992) · Neural Computation · doi:10.1162/neco.1992.4.1.1 · Cited by 3,578 · cited 1×
“Thus, as has also been discussed in [34], when the number of hidden units increases, bias falls but variance increases.”
BackgroundCraven, P., & Wahba, G. (1978) · Numerische Mathematik · doi:10.1007/bf01404567 · Cited by 3,388
Valiant, L. G. (1984) · Communications of the ACM · doi:10.1145/1968.1972 · Cited by 3,279
Pattern Classification and Scene Analysis.
Clarke, M. R. B., Duda, R. O., & Hart, P. E. (1974) · Journal of the Royal Statistical Society. Series A (General) · doi:10.2307/2344977 · Cited by 3,222
(1989) · Cited by 3,214
A practical Bayesian framework for backpropagation networks
MacKay, D. J. C. (1992) · Neural Computation · doi:10.1162/neco.1992.4.3.448 · Cited by 2,917
Universal approximation bounds for superpositions of a sigmoidal function
Barron, A. R. (1993) · IEEE Transactions on Information Theory · doi:10.1109/18.256500 · Cited by 2,804
The Cascade-Correlation learning architecture
Fahlman, S. E., & Lebiere, C. J. (2018) · doi:10.1184/r1/6610403 · Cited by 2,372
Friedman, J. H., & Stuetzle, W. (1981) · Journal of the American Statistical Association · doi:10.1080/01621459.1981.10477729 · Cited by 2,216
Reed, R. (1993) · IEEE Transactions on Neural Networks · doi:10.1109/72.248452 · Cited by 1,708
(1993) · CaltechAUTHORS (California Institute of Technology) · Cited by 1,498
Olmsted, J. M. H., Kolmogorov, A. N., Fomin, S. V., & Silverman, R. A. (1971) · The American Mathematical Monthly · doi:10.2307/2316932 · Cited by 1,397
A Resource-Allocating Network for Function Interpolation
Platt, J. (1991) · Neural Computation · doi:10.1162/neco.1991.3.2.213 · Cited by 1,354
Growing cell structures—A self-organizing network for unsupervised and supervised learning
Fritzke, B. (1994) · Neural Networks · doi:10.1016/0893-6080(94)90091-4 · Cited by 1,270
Statistical predictor identification
Akaike, H. (1970) · Annals of the Institute of Statistical Mathematics · doi:10.1007/bf02506337 · Cited by 1,250
Statistical Predictor Identification
Akaike, H. (1998) · Springer Series in Statistics · doi:10.1007/978-1-4612-1694-0_11 · Cited by 1,213
First- and Second-Order Methods for Learning: Between Steepest Descent and Newton's Method
Battiti, R. (1992) · Neural Computation · doi:10.1162/neco.1992.4.2.141 · Cited by 1,193
Neural Networks: A Review from a Statistical Perspective
Cheng, B., & Titterington, D. M. (1994) · Statistical Science · doi:10.1214/ss/1177010638 · Cited by 1,048
Mackay, D. J. C. (1995) · Network Computation in Neural Systems · doi:10.1088/0954-898x/6/3/011 · Cited by 915
Approximation and Radial-Basis-Function Networks
Park, J., & Sandberg, I. W. (1993) · Neural Computation · doi:10.1162/neco.1993.5.2.305 · Cited by 816
Exploratory Projection Pursuit
Friedman, J. H. (1987) · Journal of the American Statistical Association · doi:10.1080/01621459.1987.10478427 · Cited by 794
Layered Neural Networks with Gaussian Hidden Units as Universal Approximations
Hartman, E. J., Keeler, J. D., & Kowalski, J. M. (1990) · Neural Computation · doi:10.1162/neco.1990.2.2.210 · Cited by 782
Designing neural networks using genetic algorithms
Miller, G. F., Todd, P. M., & Hegde, S. S. (1989) · international conference on Genetic algorithms · Cited by 737
Cross-Validatory Choice and Assessment of Statistical Predictions (With Discussion)
Stone, M. (1976) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1976.tb01573.x · Cited by 686
A simple procedure for pruning back-propagation trained neural networks
Karnin, E. D. (1990) · IEEE Transactions on Neural Networks · doi:10.1109/72.80236 · Cited by 670
Neural Networks and Related Methods for Classification
Ripley, B. D. (1994) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1994.tb01990.x · Cited by 627
Self-Organizing Methods in Modeling: Gmdh Type Algorithms
Farlow, S. J. (1984) · Medical Entomology and Zoology · Cited by 593
Training a 3-node neural network is NP-complete
Blum, A. L., & Rivest, R. L. (1992) · Neural Networks · doi:10.1016/s0893-6080(05)80010-3 · Cited by 543
Skeletonization: A Technique for Trimming the Fat from a Network via Relevance Assessment
Mozer, M. C., & Smolensky, P. (1988) · neural information processing systems · Cited by 540
The Upstart Algorithm: A Method for Constructing and Training Feedforward Neural Networks
Frean, M. R. (1990) · Neural Computation · doi:10.1162/neco.1990.2.2.198 · Cited by 517
Statistical aspects of neural networks
Ripley, B. D. (1993) · Networks and Chaos — Statistical and Probabilistic Aspects · doi:10.1007/978-1-4899-3099-6_2 · Cited by 504
Back-propagation algorithm which varies the number of hidden units
Hirose, Y., Yamashita, K., & Hijiya, S. (1991) · Neural Networks · doi:10.1016/0893-6080(91)90032-z · Cited by 503
Some new results on neural network approximation
Hornik, K. (1993) · Neural Networks · doi:10.1016/s0893-6080(09)80018-x · Cited by 496
A review of evolutionary artificial neural networks
Yao, X. (1993) · International Journal of Intelligent Systems · doi:10.1002/int.4550080406 · Cited by 492
Comparing Biases for Minimal Network Construction with Back-Propagation
Hanson, S. J., & Pratt, L. (1988) · neural information processing systems · Cited by 480
Jones, L. K. (1992) · The Annals of Statistics · doi:10.1214/aos/1176348546 · Cited by 474
A Function Estimation Approach to Sequential Learning with Neural Networks
Kadirkamanathan, V., & Niranjan, M. (1993) · Neural Computation · doi:10.1162/neco.1993.5.6.954 · Cited by 458
Dynamic node creation in backpropagation networks
ASH, T. (1989) · Connection Science · doi:10.1080/09540098908915647 · Cited by 415
Approximation and estimation bounds for artificial neural networks
Barron, A. R. (1994) · Machine Learning · doi:10.1007/bf00993164 · Cited by 399
Bayesian Regularization and Pruning Using a Laplace Prior
Williams, P. M. (1995) · Neural Computation · doi:10.1162/neco.1995.7.1.117 · Cited by 377
Buntine, W., & Weigend, A. S. (1991) · Complex Systems · Cited by 361
A Gaussian potential function network with hierarchically self-organizing learning
Lee, S., & Kil, R. M. (1991) · Neural Networks · doi:10.1016/0893-6080(91)90005-p · Cited by 325
Optimization—Theory and Practice
Forst, W., & Hoffmann, D. F. (2010) · Springer undergraduate texts in mathematics and technology · doi:10.1007/978-0-387-78977-4 · Cited by 319
Shin, Y., & Ghosh, J. (2002) · IJCNN-91-Seattle International Joint Conference on Neural Networks · doi:10.1109/ijcnn.1991.155142 · Cited by 299
Regression modeling in back-propagation and projection pursuit learning
Hwang, J.-N., Lay, S.-R., Maechler, M., Martin, R. D., & Schimert, J. (1994) · IEEE Transactions on Neural Networks · doi:10.1109/72.286906 · Cited by 278
Gong, G. (1986) · Journal of the American Statistical Association · doi:10.1080/01621459.1986.10478245 · Cited by 243
Neural network design and the complexity of learning
(1991) · Choice Reviews Online · doi:10.5860/choice.28-3325 · Cited by 241
Objective functions for training new hidden units in constructive neural networks
Kwok, T.-Y., & Yeung, D.-Y. (1997) · IEEE Transactions on Neural Networks · doi:10.1109/72.623214 · Cited by 238
Self-citationNeural Network Design and the Complexity of Learning
Judd, J. S. (1990) · The MIT Press eBooks · doi:10.7551/mitpress/4932.001.0001 · Cited by 210
Use of a quasi-Newton method in a feedforward neural network construction algorithm
Setiono, R., & Hui, L. C. K. (1995) · IEEE Transactions on Neural Networks · doi:10.1109/72.363426 · Cited by 199
On a Conjecture of Huber Concerning the Convergence of Projection Pursuit Regression
Jones, L. K. (1987) · The Annals of Statistics · doi:10.1214/aos/1176350382 · Cited by 193
An Overview of Predictive Learning and Function Approximation
Friedman, J. H. (1994) · From Statistics to Neural Networks · doi:10.1007/978-3-642-79119-2_1 · Cited by 189
A review of Bayesian neural networks with an application to near infrared spectroscopy
Thodberg, H. H. (1996) · IEEE Transactions on Neural Networks · doi:10.1109/72.478392 · Cited by 184
A Probabilistic Resource Allocating Network for Novelty Detection
Roberts, S., & Tarassenko, L. (1994) · Neural Computation · doi:10.1162/neco.1994.6.2.270 · Cited by 184
Hornik, K., Stinchcombe, M., White, H., & Auer, P. (1994) · Neural Computation · doi:10.1162/neco.1994.6.6.1262 · Cited by 177
A Back-Propagation Algorithm with Optimal Use of Hidden Units
Chauvin, Y. (1988) · neural information processing systems · Cited by 168
Neural net algorithms that learn in polynomial time from examples and queries
Baum, E. B. (1991) · IEEE Transactions on Neural Networks · doi:10.1109/72.80287 · Cited by 168
Prediction Risk and Architecture Selection for Neural Networks
Moody, J. (1994) · From Statistics to Neural Networks · doi:10.1007/978-3-642-79119-2_7 · Cited by 159
Shin, Y., & Ghosh, J. (1995) · IEEE Transactions on Neural Networks · doi:10.1109/72.377967 · Cited by 149
Nonparametric estimation via empirical risk minimization
Lugosi, G., & Zeger, K. (1995) · IEEE Transactions on Information Theory · doi:10.1109/18.382014 · Cited by 147
Xu, L., Krzyżak, A., & Yuille, A. (1994) · Neural Networks · doi:10.1016/0893-6080(94)90040-x · Cited by 141
Note on generalization, regularization and architecture selection in nonlinear learning systems
Moody, J. E. (2002) · Machine learning for signal processing ... · doi:10.1109/nnsp.1991.239541 · Cited by 138
A Convergence Theorem for Sequential Learning in Two-Layer Perceptrons
Marchand, M., Golea, M., & Ruján, P. (1990) · Europhysics Letters (EPL) · doi:10.1209/0295-5075/11/6/001 · Cited by 133
Multidimensional Additive Spline Approximation
Friedman, J. H., Grosse, E., & Stuetzle, W. (1983) · SIAM Journal on Scientific and Statistical Computing · doi:10.1137/0904023 · Cited by 103
Self-organizing network for optimum supervised learning
Tenorio, M. F., & Lee, W.-T. (1990) · IEEE Transactions on Neural Networks · doi:10.1109/72.80209 · Cited by 94
Toward generating neural network structures for function approximation
Nabhan, T. M., & Zomaya, A. Y. (1994) · Neural Networks · doi:10.1016/0893-6080(94)90058-2 · Cited by 86
Complexity Results on Learning by Neural Nets
Lin, J.-H., & Vitter, J. S. (1991) · Machine Learning · doi:10.1007/bf00114777 · Cited by 85
A Proposal for More Powerful Learning Algorithms
Baum, E. B. (1989) · Neural Computation · doi:10.1162/neco.1989.1.2.201 · Cited by 73
Predicting the Future: Advantages of Semilocal Units
Hartman, E., & Keeler, J. D. (1991) · Neural Computation · doi:10.1162/neco.1991.3.4.566 · Cited by 73
Statistical Ideas for Selecting Network Architectures
Ripley, B. D. (1995) · Springer eBooks · doi:10.1007/978-1-4471-3087-1_36 · Cited by 66
Dynamic node architecture learning: An information theoretic approach
Bartlett, E. B. (1994) · Neural Networks · doi:10.1016/0893-6080(94)90061-2 · Cited by 65
Comparative Bibliography of Ontogenic Neural Networks
Fiesler, E. (1994) · Springer eBooks · doi:10.1007/978-1-4471-2097-1_188 · Cited by 62
Automatic Smoothing Spline Projection Pursuit
Roosen, C. B., & Hastie, T. J. (1994) · Journal of Computational and Graphical Statistics · doi:10.1080/10618600.1994.10474642 · Cited by 54
Radial basis function network for speech pattern classification
Renals, S. J. (1989) · Electronics Letters · doi:10.1049/el:19890300 · Cited by 52
Šı́ma, J. (1994) · Neural Computation · doi:10.1162/neco.1994.6.5.842 · Cited by 47
Recursive dynamic node creation in multilayer neural networks
Azimi-Sadjadi, M. R., Sheedvash, S., & Trujillo, F. O. (1993) · IEEE Transactions on Neural Networks · doi:10.1109/72.207612 · Cited by 45
An incremental learning algorithm that optimizes network size and sample size in one trial
Zhang, B. (1994) · IEEE International Conference on Neural Networks/IEEE ... International Conference on Neural Networks · doi:10.1109/icnn.1994.374165 · Cited by 39
Controlled Growth of Cascade Correlation Nets
Hansen, L. K., & Pedersen, M. W. (1994) · Springer eBooks · doi:10.1007/978-1-4471-2097-1_189 · Cited by 37
Intrator, N. (1993) · Neural Computation · doi:10.1162/neco.1993.5.3.443 · Cited by 37
Neural Networks and Related Methods for Classification - Discussion
Whittle, P., Kay, J., Hand, D. J., Tarassenko, L., Brown, P., Titterington, D. M., Taylor, C. R., Gilks, W. R., Critchley, F., Mayne, A. J., et al. (1994) · Journal of the Royal Statistical Society Series B (Methodological) · Cited by 35
Use of bias term in projection pursuit learning improves approximation and convergence properties
Kwok, T.-Y., & Yeung, D. (1996) · IEEE Transactions on Neural Networks · doi:10.1109/72.536312 · Cited by 35
Self-citationLittmann, E., & Ritter, H. J. (2003) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.1992.226955 · Cited by 33
Optimization of the hidden unit function in feedforward neural networks
Fujita, O. (1992) · Neural Networks · doi:10.1016/s0893-6080(05)80136-4 · Cited by 30
Pattern classification using projection pursuit
Flick, T. E., Jones, L. K., Priest, R. G., & Herman, C. (1990) · Pattern Recognition · doi:10.1016/0031-3203(90)90083-w · Cited by 30
Bayesian regularization in constructive neural networks
Kwok, T.-Y., & Yeung, D. (1996) · Lecture notes in computer science · doi:10.1007/3-540-61510-5_95 · Cited by 28
Self-citationSaha, A., Wu, C.-L., & Tang, D.-S. (1993) · IEEE Transactions on Computers · doi:10.1109/12.257708 · Cited by 28
Convergence properties of cascade correlation in function approximation
Drago, G. P., & Ridella, S. (1994) · Neural Computing and Applications · doi:10.1007/bf01415010 · Cited by 14
Forecasting Demand for Electric Power
Yuan, J.-L., & Fine, T. L. (1992) · Cited by 14
Coarse Coding Resource-Allocating Network
Deco, G., & Ebmeyer, J. (1993) · Neural Computation · doi:10.1162/neco.1993.5.1.105 · Cited by 13
Network complexity and learning efficiency of constructive learning algorithms
Fang, W., & Lacher, R. C. (1994) · doi:10.1109/icnn.1994.374191 · Cited by 12
Neural networks and nonparametric regression
Cherkassky, V. (2003) · Machine learning for signal processing ... · doi:10.1109/nnsp.1992.253661 · Cited by 8
A training approach based on linear separability analysis for layered perceptrons
Zhang, D. D., Kamel, M. S., & Elmasry, M. I. (1994) · doi:10.1109/icnn.1994.374217 · Cited by 3
A taxonomy of neural network optimality
Nelson, D. E., & Rogers, S. K. (2003) · IEEE National Aerospace and Electronics Conference (NAECON) · doi:10.1109/naecon.1992.220489 · Cited by 3
Valiant, L. G. (2026) · Foundations of Computation and Machine Learning · doi:10.1145/3749740.3749746
Cited by
The graph neural network model
Scarselli, F., Gori, M., Tsoi, A. C., Hagenbuchner, M., & Monfardini, G. (2009) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2008.2005605 · Cited by 9,582
Maier, H. R., & Dandy, G. C. (2000) · Environmental Modelling & Software · doi:10.1016/s1364-8152(99)00007-9 · Cited by 2,377
Trends in extreme learning machines: A review
Huang, G., Huang, G.-B., Song, S., & You, K. (2015) · Neural Networks · doi:10.1016/j.neunet.2014.10.001 · Cited by 1,732
Hydrological modelling using artificial neural networks
Dawson, C. W., & Wilby, R. L. (2001) · Progress in Physical Geography: Earth and Environment · doi:10.1177/030913330102500104 · Cited by 903
Artificial neural networks for rf and microwave design-from theory to practice
Zhang, Q.-J., Gupta, K. C., & Devabhaktuni, V. K. (2003) · IEEE Transactions on Microwave Theory and Techniques · doi:10.1109/tmtt.2003.809179 · Cited by 749
Metadata
- Authors
- Kwok, T.-Y., & Yeung, D.-Y.
- Venue
- IEEE Transactions on Neural Networks
- Published
- 1997-05
- Volume
- 8
- Issue
- 3
- Pages
- 630-645
- ISSN
- 1045-9227 (ISSN-L)
- 1941-0093
- OpenAlex
- W2117941247
- Cited by
- 493
- References
- 108
- FWCI
- 13.79
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.