The DOI matches its Crossref record; 301 of the 301 references it cites were found in a registry or an index.
Metaheuristic design of feedforward neural networks: A review of two decades of research
Open accessAbstract
Over the past two decades, the feedforward neural network (FNN) optimization has been a key interest among the researchers and practitioners of multiple disciplines. The FNN optimization is often viewed from the various perspectives: the optimization of weights, network architecture, activation nodes, learning parameters, learning environment, etc. Researchers adopted such different viewpoints mainly to improve the FNN's generalization ability. The gradient-descent algorithm such as backpropagation has been widely applied to optimize the FNNs. Its success is evident from the FNN's application to numerous real-world problems. However, due to the limitations of the gradient-based optimization methods, the metaheuristic algorithms including the evolutionary algorithms, swarm intelligence, etc., are still being widely explored by the researchers aiming to obtain generalized FNN for a given problem. This article attempts to summarize a broad spectrum of FNN optimization methodologies including conventional and metaheuristic approaches. This article also tries to connect various research directions emerged out of the FNN optimization practices, such as evolving neural network (NN), cooperative coevolution NN, complex-valued NN, deep learning, extreme learning machine, quantum NN, etc. Additionally, it provides interesting research challenges for future research to cope-up with the present information processing era.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Cortes, C., & Vapnik, V. (1995) · Machine Learning · doi:10.1007/bf00994018 · Cited by 59,421
Holland, J. H. (1992) · The MIT Press eBooks · doi:10.7551/mitpress/1090.001.0001 · Cited by 47,002
Optimization by Simulated Annealing
Kirkpatrick, S., Gelatt, C. D., & Vecchi, M. P. (1983) · Science · doi:10.1126/science.220.4598.671 · Cited by 45,134
Equation of State Calculations by Fast Computing Machines
Metropolis, N., Rosenbluth, A. W., Rosenbluth, M. N., Teller, A. H., & Teller, E. (1953) · The Journal of Chemical Physics · doi:10.1063/1.1699114 · Cited by 37,192
Learning representations by back-propagating errors
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986) · Nature · doi:10.1038/323533a0 · Cited by 31,892
An Algorithm for Least-Squares Estimation of Nonlinear Parameters
Marquardt, D. W. (1963) · Journal of the Society for Industrial and Applied Mathematics · doi:10.1137/0111030 · Cited by 30,728
Storn, R., & Price, K. (1997) · Journal of Global Optimization · doi:10.1023/a:1008202821328 · Cited by 29,569
Exploration and Exploitation in Organizational Learning
March, J. G. (1991) · Organization Science · doi:10.1287/orsc.2.1.71 · Cited by 21,942
Multilayer feedforward networks are universal approximators
Hornik, K., Stinchcombe, M., & White, H. (1989) · Neural Networks · doi:10.1016/0893-6080(89)90020-8 · Cited by 21,728
Reducing the dimensionality of data with neural networks
Hinton, G. E., & Salakhutdinov, R. R. (2006) · Science · doi:10.1126/science.1127647 · Cited by 20,928
Mirjalili, S., Mirjalili, S. M., & Lewis, A. (2014) · Advances in Engineering Software · doi:10.1016/j.advengsoft.2013.12.007 · Cited by 19,343
Neural networks and physical systems with emergent collective computational abilities.
Hopfield, J. J. (1982) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.79.8.2554 · Cited by 19,270
Deep learning in neural networks: an overview
Schmidhuber, J. (2015) · Neural Networks · doi:10.1016/j.neunet.2014.09.003 · Cited by 18,570
A logical calculus of the ideas immanent in nervous activity
McCulloch, W. S., & Pitts, W. (1943) · The Bulletin of Mathematical Biophysics · doi:10.1007/bf02478259 · Cited by 18,566
A fast learning algorithm for deep belief nets
Hinton, G. E., Osindero, S., & Teh, Y.-W. (2006) · Neural Computation · doi:10.1162/neco.2006.18.7.1527 · Cited by 16,336
Breiman, L. (1996) · Machine Learning · doi:10.1007/bf00058655 · Cited by 16,325
A new optimizer using particle swarm theory
Eberhart, R., & Kennedy, J. (2002) · MHS'95. Proceedings of the Sixth International Symposium on Micro Machine and Human Science · doi:10.1109/mhs.1995.494215 · Cited by 16,067
No free lunch theorems for optimization
Wolpert, D. H., & Macready, W. G. (1997) · IEEE Transactions on Evolutionary Computation · doi:10.1109/4235.585893 · Cited by 14,237
Approximation by superpositions of a sigmoidal function
Cybenko, G. (1989) · Mathematics of Control, Signals, and Systems · doi:10.1007/bf02551274 · Cited by 13,883
Extreme learning machine: Theory and applications
Huang, G.-B., Zhu, Q.-Y., & Siew, C.-K. (2006) · Neurocomputing · doi:10.1016/j.neucom.2005.12.126 · Cited by 13,285
The perceptron: A probabilistic model for information storage and organization in the brain.
Rosenblatt, F. (1958) · Psychological Review · doi:10.1037/h0042519 · Cited by 12,280
Ant system: optimization by a colony of cooperating agents
Dorigo, M., Maniezzo, V., & Colorni, A. (1996) · IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) · doi:10.1109/3477.484436 · Cited by 12,067
Bertsekas, D. P. (1997) · Journal of the Operational Research Society · doi:10.1038/sj.jors.2600425 · Cited by 11,726
Hinton, G., Deng, L., Yu, D., Dahl, G., Mohamed, A.-R., Jaitly, N., Senior, A., Vanhoucke, V., Nguyen, P., Sainath, T., et al. (2012) · IEEE Signal Processing Magazine · doi:10.1109/msp.2012.2205597 · Cited by 10,260
A modified particle swarm optimizer
Shi, Y., & Eberhart, R. (2002) · 1998 IEEE International Conference on Evolutionary Computation Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98TH8360) · doi:10.1109/icec.1998.699146 · Cited by 10,229
Self-organized formation of topologically correct feature maps
Kohonen, T. (1982) · Biological Cybernetics · doi:10.1007/bf00337288 · Cited by 9,529
Multilayer feedforward networks are universal approximators
HornikK., StinchcombeM., & WhiteH. (1989) · Neural Networks · doi:10.5555/70405.70408 · Cited by 9,081
Reinforcement Learning: A Survey
Kaelbling, L. P., Littman, M. L., & Moore, A. W. (1996) · Journal of Artificial Intelligence Research · doi:10.1613/jair.301 · Cited by 8,972
Methods of conjugate gradients for solving linear systems
Hestenes, M. R., & Stiefel, E. (1952) · Journal of Research of the National Bureau of Standards · doi:10.6028/jres.049.044 · Cited by 8,132
An introduction to computing with neural nets
Lippmann, R. (1987) · IEEE ASSP Magazine · doi:10.1109/massp.1987.1165576 · Cited by 7,825
Training feedforward networks with the Marquardt algorithm
Hagan, M. T., & Menhaj, M. B. (1994) · IEEE Transactions on Neural Networks · doi:10.1109/72.329697 · Cited by 7,701
Neural Networks And Learning Machines
Haykin, S. (2010) · Cited by 7,535
A systematic analysis of performance measures for classification tasks
Sokolova, M., & Lapalme, G. (2009) · Information Processing & Management · doi:10.1016/j.ipm.2009.03.002 · Cited by 7,515
GSA: A Gravitational Search Algorithm
Rashedi, E., Nezamabadi-pour, H., & Saryazdi, S. (2009) · Information Sciences · doi:10.1016/j.ins.2009.03.004 · Cited by 7,358
Statistical pattern recognition: a review
Jain, A. K., Duin, P. W., & Mao, J. (2000) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.824819 · Cited by 6,767
Pencina, M. J., D' Agostino, R. B., D' Agostino, R. B., & Vasan, R. S. (2008) · Statistics in Medicine · doi:10.1002/sim.2929 · Cited by 6,493
A New Heuristic Optimization Algorithm: Harmony Search
Geem, Z. W., Kim, J. H., & Loganathan, G. V. (2001) · SIMULATION · doi:10.1177/003754970107600201 · Cited by 6,247
Cuckoo Search via Lévy flights
Yang, X.-S., & Deb, S. (2009) · 2009 World Congress on Nature & Biologically Inspired Computing (NaBIC) · doi:10.1109/nabic.2009.5393690 · Cited by 6,185
Approximation capabilities of multilayer feedforward networks
Hornik, K. (1991) · Neural Networks · doi:10.1016/0893-6080(91)90009-t · Cited by 6,091
Practical methods of optimization
(1988) · Mathematics and Computers in Simulation · doi:10.1016/0378-4754(88)90129-2 · Cited by 5,818
Glover, F. (1990) · ORSA Journal on Computing · doi:10.1287/ijoc.2.1.4 · Cited by 5,647
AN IDEA BASED ON HONEY BEE SWARM FOR NUMERICAL OPTIMIZATION
Karaboğa, D. (2005) · Cited by 5,466
Differential Evolution: A Survey of the State-of-the-Art
Das, S., & Suganthan, P. N. (2011) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2010.2059031 · Cited by 5,307
A Fast Elitist Non-dominated Sorting Genetic Algorithm for Multi-objective Optimization: NSGA-II
Deb, K., Agrawal, S., Pratap, A., & Meyarivan, T. (2000) · Lecture Notes in Computer Science · doi:10.1007/3-540-45356-3_83 · Cited by 5,289
Diebold, F., & Mariano, R. (1994) · National Bureau of Economic Research · doi:10.3386/t0169 · Cited by 5,185
Diebold, F. X., & Mariano, R. S. (1995) · Journal of Business & Economic Statistics · doi:10.1080/07350015.1995.10524599 · Cited by 5,130
Glover, F. (1989) · ORSA Journal on Computing · doi:10.1287/ijoc.1.3.190 · Cited by 5,106
Artificial neural networks: a tutorial
Jain, A. K., Mao, J., & Mohiuddin, K. M. (1996) · Computer · doi:10.1109/2.485891 · Cited by 4,946
A discrete binary version of the particle swarm algorithm
Kennedy, J., & Eberhart, R. C. (2002) · 1997 IEEE International Conference on Systems, Man, and Cybernetics. Computational Cybernetics and Simulation · doi:10.1109/icsmc.1997.637339 · Cited by 4,767
The Strength of Weak Learnability
Schapire, R. E. (1990) · Machine Learning · doi:10.1007/bf00116037 · Cited by 4,600
Hansen, L. K., & Salamon, P. (1990) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.58871 · Cited by 4,260
Mladenović, N., & Hansen, P. (1997) · Computers & Operations Research · doi:10.1016/s0305-0548(97)00031-2 · Cited by 4,254
WIDROW, B., & HOFF, M. E. (1960) · doi:10.21236/ad0241531 · Cited by 4,009
A direct adaptive method for faster backpropagation learning: the RPROP algorithm
Riedmiller, M., & Braun, H. (2002) · IEEE International Conference on Neural Networks · doi:10.1109/icnn.1993.298623 · Cited by 3,917
Perceptrons: An Introduction to Computational Geometry
Minsky, M., & Papert, S. A. (1969) · Cited by 3,718
Practical Methods of Optimization.
Witzgall, C., & Fletcher, R. (1989) · Mathematics of Computation · doi:10.2307/2008742 · Cited by 3,591
Neural Networks and the Bias/Variance Dilemma
Geman, S., Bienenstock, E., & Doursat, R. (1992) · Neural Computation · doi:10.1162/neco.1992.4.1.1 · Cited by 3,578 · cited 1×
“To achieve generalization, FNNs need to avoid both underfitting and overfitting during training, which is associated with high statistical bias and high statistical variance [54].”
BackgroundVoxNet: A 3D Convolutional Neural Network for real-time object recognition
Maturana, D., & Scherer, S. (2015) · 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2015.7353481 · Cited by 3,458
Radial Basis Functions, Multi-Variable Functional Interpolation and Adaptive Networks
Broomhead, D. S., & Lowe, D. (1988) · OpenGrey (Institut de l'Information Scientifique et Technique) · Cited by 3,443
Thermodynamical approach to the traveling salesman problem: An efficient simulation algorithm
Černý, V. (1985) · Journal of Optimization Theory and Applications · doi:10.1007/bf00940812 · Cited by 3,436
A learning algorithm for Boltzmann machines
Ackley, D. H., Hinton, G. E., & Sejnowski, T. J. (1985) · Cognitive Science · doi:10.1207/s15516709cog0901_7 · Cited by 3,402
Evolving Neural Networks through Augmenting Topologies
Stanley, K. O., & Miikkulainen, R. (2002) · Evolutionary Computation · doi:10.1162/106365602320169811 · Cited by 3,258
Genetic Algorithms and Machine Learning
Goldberg, D. E., & Holland, J. H. (1988) · Machine Learning · doi:10.1023/a:1022602019183 · Cited by 3,193
Biomimicry of bacterial foraging for distributed optimization and control
(2002) · IEEE Control Systems · doi:10.1109/mcs.2002.1004010 · Cited by 3,069
Self-organized formation of topographically correct feature maps
Kohonen, T. (1982) · Biological Cybernetics · Cited by 2,995
Evolving artificial neural networks
Yao, X. (1999) · Proceedings of the IEEE · doi:10.1109/5.784219 · Cited by 2,981
Barto, A. G. (1998) · IFAC Proceedings Volumes · doi:10.1016/s1474-6670(17)38315-5 · Cited by 2,976
Ensemble based systems in decision making
Polikar, R. (2006) · IEEE Circuits and Systems Magazine · doi:10.1109/mcas.2006.1688199 · Cited by 2,938
Beyond Regression : "New Tools for Prediction and Analysis in the Behavioral Sciences
Werbos, P. J. (1974) · Medical Entomology and Zoology · Cited by 2,789
Greedy Randomized Adaptive Search Procedures
Feo, T. A., & Resende, M. G. C. (1995) · Journal of Global Optimization · doi:10.1007/bf01096763 · Cited by 2,778
Beyond regression : new fools for prediction and analysis in the behavioral sciences
Werbos, P. J. (1974) · Medical Entomology and Zoology · Cited by 2,739
Firefly algorithm, stochastic test functions and design optimisation
Yang, X. S. (2010) · International Journal of Bio-Inspired Computation · doi:10.1504/ijbic.2010.032124 · Cited by 2,729
The particle swarm optimization algorithm: convergence analysis and parameter selection
Trelea, I. C. (2003) · Information Processing Letters · doi:10.1016/s0020-0190(02)00447-7 · Cited by 2,644
Universal Approximation using Incremental Constructive Feedforward Networks with Random Hidden Nodes
Huang, G.-B., Chen, L., & Siew, C.-K. (2006) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2006.875977 · Cited by 2,614
Weik, M. H. (2000) · Computer Science and Communications Dictionary · doi:10.1007/1-4020-0613-6_12436 · Cited by 2,570
Kalman Filtering and Neural Networks
(2001) · Wiley eBooks · doi:10.1002/0471221546 · Cited by 2,386
The Cascade-Correlation learning architecture
Fahlman, S. E., & Lebiere, C. J. (2018) · doi:10.1184/r1/6610403 · Cited by 2,372
Greedy Randomized Adaptive Search Procedures
Resende, M. G. C., & Ribeiro, C. C. (2006) · International Series in Operations Research & Management Science · doi:10.1007/0-306-48056-5_8 · Cited by 2,310
13 Flower Pollination Algorithm for Global OptimizationPreprint
Yang, X. (2012) · Cited by 2,272
Multiobjective evolutionary algorithms: A survey of the state of the art
Zhou, A., Qu, B.-Y., Li, H., Zhao, S.-Z., Suganthan, P. N., & Zhang, Q. (2011) · Swarm and Evolutionary Computation · doi:10.1016/j.swevo.2011.03.001 · Cited by 2,255
30 years of adaptive neural networks: perceptron, Madaline, and backpropagation
Widrow, B., & Lehr, M. A. (1990) · Proceedings of the IEEE · doi:10.1109/5.58323 · Cited by 2,242
Leshno, M., Lin, V. Y., Pinkus, A., & Schocken, S. (1993) · Neural Networks · doi:10.1016/s0893-6080(05)80131-5 · Cited by 2,121
A Cooperative Approach to Particle Swarm Optimization
vandenBergh, F., & Engelbrecht, A. P. (2004) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2004.826069 · Cited by 2,120
Evaluating the predictive performance of habitat models developed using logistic regression
Pearce, J., & Ferrier, S. (2000) · Ecological Modelling · doi:10.1016/s0304-3800(00)00322-7 · Cited by 2,002
Ensembling neural networks: Many could be better than all
Zhou, Z.-H., Wu, J., & Tang, W. (2002) · Artificial Intelligence · doi:10.1016/s0004-3702(02)00190-x · Cited by 1,964
Estimation of Distribution Algorithms: A New Tool for Evolutionary Computation
Larraanaga, P., & Lozano, J. A. (2001) · Kluwer Academic Publishers eBooks · Cited by 1,914
An improved harmony search algorithm for solving optimization problems
Mahdavi, M., Fesanghary, M., & Damangir, E. (2007) · Applied Mathematics and Computation · doi:10.1016/j.amc.2006.11.033 · Cited by 1,863
Increased rates of convergence through learning rate adaptation
Jacobs, R. A. (1988) · Neural Networks · doi:10.1016/0893-6080(88)90003-2 · Cited by 1,806
Pipino, L. L., Lee, Y. W., & Wang, R. Y. (2002) · Communications of the ACM · doi:10.1145/505248.506010 · Cited by 1,724
Equation of state calculations by fast computing machines
Metropolis, N., Rosenbluth, A. W., Rosenbluth, M. N., Teller, A. H., & Teller, E. (1953) · doi:10.2172/4390578 · Cited by 1,701
Neural networks for classification: a survey
Zhang, G. P. (2000) · IEEE Transactions on Systems, Man and Cybernetics, Part C (Applications and Reviews) · doi:10.1109/5326.897072 · Cited by 1,667
A survey on optimization metaheuristics
Boussaïd, I., Lepagnot, J., & Siarry, P. (2013) · Information Sciences · doi:10.1016/j.ins.2013.02.041 · Cited by 1,655
The Lack of A Priori Distinctions Between Learning Algorithms
Wolpert, D. H. (1996) · Neural Computation · doi:10.1162/neco.1996.8.7.1341 · Cited by 1,647
Flower Pollination Algorithm for Global Optimization
Yang, X.-S. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-32894-7_27 · Cited by 1,630
On Evolution, Search, Optimization, Genetic Algorithms and Martial Arts : Towards Memetic Algorithms
Moscato, P. (1989) · Cited by 1,516
Parallel Networks that Learn to Pronounce English Text
B, T. J. S. A. (1987) · Cited by 1,514
Anchoring Data Quality Dimensions in Ontological Foundations
Wand, Y., & Wang, R. Y. (1996) · Communications of the ACM · doi:10.1145/240455.240479 · Cited by 1,479
Competitive learning: From interactive activation to adaptive resonance
Grossberg, S. (1987) · Cognitive Science · doi:10.1111/j.1551-6708.1987.tb00862.x · Cited by 1,471
Understanding Big Data: Analytics for Enterprise Class Hadoop and Streaming Data
Zikopoulos, P., & Eaton, C. (2011) · Cited by 1,455
Extreme Learning Machine for Multilayer Perceptron
Tang, J., Deng, C., & Huang, G.-B. (2016) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2015.2424995 · Cited by 1,451
Ant colony optimization for continuous domains
Socha, K., & Dorigo, M. (2008) · European Journal of Operational Research · doi:10.1016/j.ejor.2006.06.046 · Cited by 1,414
A Nonlinear Conjugate Gradient Method with a Strong Global Convergence Property
Dai, Y. H., & Yuan, Y. (1999) · SIAM Journal on Optimization · doi:10.1137/s1052623497318992 · Cited by 1,378
A Comprehensive Survey of Evolutionary-Based Multiobjective Optimization Techniques
Coello Coello, C. A. (1999) · Knowledge and Information Systems · doi:10.1007/bf03325101 · Cited by 1,367
Regularization Theory and Neural Networks Architectures
Girosi, F., Jones, M., & Poggio, T. (1995) · Neural Computation · doi:10.1162/neco.1995.7.2.219 · Cited by 1,349
Training with Noise is Equivalent to Tikhonov Regularization
Bishop, C. M. (1995) · Neural Computation · doi:10.1162/neco.1995.7.1.108 · Cited by 1,325
Das, I., & Dennis, J. E. (1997) · Structural Optimization · doi:10.1007/bf01197559 · Cited by 1,208
A Survey of Dimension Reduction Techniques
Fodor, I. K. (2002) · doi:10.2172/15002155 · Cited by 1,116
Methods of integrating data to uncover genotype-phenotype interactions
Ritchie, M. D., Holzinger, E. R., Li, R., Pendergrass, S. A., & Kim, D. (2015) · Nature Reviews Genetics · doi:10.1038/nrg3868 · Cited by 1,085
Feature Discovery by Competitive Learning*
Rumelhart, D. E., & Zipser, D. (1985) · Cognitive Science · doi:10.1207/s15516709cog0901_5 · Cited by 1,057
Metaheuristics—the metaphor exposed
Sörensen, K. (2015) · International Transactions in Operational Research · doi:10.1111/itor.12001 · Cited by 1,051
An evolutionary algorithm that constructs recurrent neural networks
Angeline, P. J., Saunders, G. M., & Pollack, J. B. (1994) · IEEE Transactions on Neural Networks · doi:10.1109/72.265960 · Cited by 1,012
Training feedforward neural networks using genetic algorithms
Montana, D. J., & Davis, L. D. (1989) · International Joint Conference on Artificial Intelligence · Cited by 1,006
Whitley, L. D. (1989) · international conference on Genetic algorithms · Cited by 992
Automatic early stopping using cross validation: quantifying the criteria
Prechelt, L. (1998) · Neural Networks · doi:10.1016/s0893-6080(98)00010-0 · Cited by 990
On Early Stopping in Gradient Descent Learning
Yao, Y., Rosasco, L., & Caponnetto, A. (2007) · Constructive Approximation · doi:10.1007/s00365-006-0663-2 · Cited by 970
A Hybrid of Genetic Algorithm and Particle Swarm Optimization for Recurrent Network Design
Juang, C.-F. (2004) · IEEE Transactions on Systems, Man and Cybernetics, Part B (Cybernetics) · doi:10.1109/tsmcb.2003.818557 · Cited by 964
An Insight into Extreme Learning Machines: Random Neurons, Random Features and Kernels
Huang, G.-B. (2014) · Cognitive Computation · doi:10.1007/s12559-014-9255-2 · Cited by 957
The self-organizing exploratory pattern of the argentine ant
Deneubourg, J. L., Aron, S., Goss, S., & Pasteels, J. M. (1990) · Journal of Insect Behavior · doi:10.1007/bf01417909 · Cited by 921
Lin, S.-W., Ying, K.-C., Chen, S.-C., & Lee, Z.-J. (2008) · Expert Systems with Applications · doi:10.1016/j.eswa.2007.08.088 · Cited by 895
Feature discovery by competitive learning
RUMELHART, D., & ZIPSER, D. (1985) · Cognitive Science · doi:10.1016/s0364-0213(85)80010-0 · Cited by 882
A new evolutionary system for evolving artificial neural networks
Yao, X., & Liu, Y. (1997) · IEEE Transactions on Neural Networks · doi:10.1109/72.572107 · Cited by 874
An empirical study of learning speed in back-propagation networks
Fahlman, S. E. (2018) · doi:10.1184/r1/6603266 · Cited by 829
Real-world Data is Dirty: Data Cleansing and The Merge/Purge Problem
Hernández, M. A., & Stolfo, S. J. (1998) · Data Mining and Knowledge Discovery · doi:10.1023/a:1009761603038 · Cited by 812
An optimizing BP neural network algorithm based on genetic algorithm
Ding, S., Su, C., & Yu, J. (2011) · Artificial Intelligence Review · doi:10.1007/s10462-011-9208-z · Cited by 802
Learning in Nonstationary Environments: A Survey
Ditzler, G., Roveri, M., Alippi, C., & Polikar, R. (2015) · IEEE Computational Intelligence Magazine · doi:10.1109/mci.2015.2471196 · Cited by 793
Tuning of the structure and parameters of a neural network using an improved genetic algorithm
Leung, F. H. F., Lam, H. K., Ling, S. H., & Tam, P. K. S. (2003) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2002.804317 · Cited by 784
Genetic Programming III: Darwinian Invention and Problem Solving
Koza, J. R., André, D., Bennett, F. H., & Keane, M. A. (1999) · Medical Entomology and Zoology · Cited by 763
Learning Representations by Back-Propagating Errors
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (2002) · Cognitive Modeling · doi:10.7551/mitpress/1888.003.0013 · Cited by 759
Genetic algorithms and neural networks: optimizing connections and connectivity
Whitley, D., Starkweather, T., & Bogart, C. (1990) · Parallel Computing · doi:10.1016/0167-8191(90)90086-o · Cited by 747
Kolmogorov's theorem and multilayer neural networks
Kůrková, V. (1992) · Neural Networks · doi:10.1016/0893-6080(92)90012-8 · Cited by 747
Mazurowski, M. A., Habas, P. A., Zurada, J. M., Lo, J. Y., Baker, J. A., & Tourassi, G. D. (2008) · Neural Networks · doi:10.1016/j.neunet.2007.12.031 · Cited by 737
Ensemble learning via negative correlation
Liu, Y., & Yao, X. (1999) · Neural Networks · doi:10.1016/s0893-6080(99)00073-8 · Cited by 712
Zhang, J.-R., Zhang, J., Lok, T.-M., & Lyu, M. R. (2007) · Applied Mathematics and Computation · doi:10.1016/j.amc.2006.07.025 · Cited by 706
Human Activity Recognition and Pattern Discovery
Kim, E., Helal, S., & Cook, D. (2010) · IEEE Pervasive Computing · doi:10.1109/mprv.2010.7 · Cited by 704
Approximation by superpositions of a sigmoidal function
Cybenko, G. (1992) · Mathematics of Control, Signals, and Systems · doi:10.1007/bf02134016 · Cited by 703
Metaheuristics: A bibliography
Osman, I. H., & Laporte, G. (1996) · Annals of Operations Research · doi:10.1007/bf02125421 · Cited by 672
On the problem of local minima in backpropagation
Gori, M., & Tesi, A. (1992) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.107014 · Cited by 650
Designing Neural Networks Using Genetic Algorithms with Graph Generation System
Kitnano, H. (1990) · Complex Systems · Cited by 612
Creating artificial neural networks that generalize
Sietsma, J., & Dow, R. J. F. (1991) · Neural Networks · doi:10.1016/0893-6080(91)90033-2 · Cited by 597
Differential Evolution Training Algorithm for Feed-Forward Neural Networks
Ilonen, J., Kamarainen, J.-K., & Lampinen, J. (2003) · Neural Processing Letters · doi:10.1023/a:1022995128597 · Cited by 582
Clever Algorithms: Nature-Inspired Programming Recipes
Brownlee, J. (2011) · Medical Entomology and Zoology · Cited by 573
Huang, G.-B., & Babri, H. A. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/72.655045 · Cited by 538
Complex-Valued Neural Networks
Hirose, A. (2012) · Studies in Computational Intelligence · doi:10.1007/978-3-642-27632-3 · Cited by 531
Comparing the predictive accuracy of models using a simple randomization test
van der Voet, H. (1994) · Chemometrics and Intelligent Laboratory Systems · doi:10.1016/0169-7439(94)85050-x · Cited by 522
The Upstart Algorithm: A Method for Constructing and Training Feedforward Neural Networks
Frean, M. R. (1990) · Neural Computation · doi:10.1162/neco.1990.2.2.198 · Cited by 517
Mirjalili, S., Mohd Hashim, S. Z., & Moradian Sardroudi, H. (2012) · Applied Mathematics and Computation · doi:10.1016/j.amc.2012.04.069 · Cited by 505
Bousquet, O., Elisseeff, A., & Ron, D. (2001) · MPG.PuRe (Max Planck Society) · Cited by 493
A review of evolutionary artificial neural networks
Yao, X. (1993) · International Journal of Intelligent Systems · doi:10.1002/int.4550080406 · Cited by 492
Pareto-Based Multiobjective Machine Learning: An Overview and Case Studies
Jin, Y., & Sendhoff, B. (2008) · IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews) · doi:10.1109/tsmcc.2008.919172 · Cited by 455
Evolutionary ensembles with negative correlation learning
Higuchi, T., Yao, X., & Liu, Y. (2000) · IEEE Transactions on Evolutionary Computation · doi:10.1109/4235.887237 · Cited by 445
Genetic evolution of the topology and weight distribution of neural networks
Maniezzo, V. (1994) · IEEE Transactions on Neural Networks · doi:10.1109/72.265959 · Cited by 436
Rhee, M., & Kim, T. (2016) · The Palgrave Encyclopedia of Strategic Management · doi:10.1057/978-1-349-94848-2_388-1 · Cited by 435
The general inefficiency of batch training for gradient descent learning
Wilson, D. R., & Martinez, T. R. (2003) · Neural Networks · doi:10.1016/s0893-6080(03)00138-2 · Cited by 418
The self-adaptive Pareto differential evolution algorithm
Abbass, H. A. (2003) · Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600) · doi:10.1109/cec.2002.1007033 · Cited by 405
Evolutionary Computation: The Fossil Record
Fogel, D. B. (1998) · Cited by 395
Feature Article—The Ellipsoid Method: A Survey
Bland, R. G., Goldfarb, D., & Todd, M. J. (1981) · Operations Research · doi:10.1287/opre.29.6.1039 · Cited by 389
A Brief Review of Nature-Inspired Algorithms for Optimization
Fister, I., Yang, X., Fister, I., Brest, J., & Fister, D. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1307.4186 · Cited by 388
Conjugate gradient algorithm for efficient training of artificial neural networks
Charalambous, C. (1992) · IEE Proceedings G Circuits Devices and Systems · doi:10.1049/ip-g-2.1992.0050 · Cited by 375
A self-adaptive global best harmony search algorithm for continuous optimization problems
Pan, Q.-K., Suganthan, P. N., Tasgetiren, M. F., & Liang, J. J. (2010) · Applied Mathematics and Computation · doi:10.1016/j.amc.2010.01.088 · Cited by 375
Validating and comparing predictive models
Baranyi, J., Pin, C., & Ross, T. (1999) · International Journal of Food Microbiology · doi:10.1016/s0168-1605(99)00035-5 · Cited by 372
Asymptotic statistical theory of overtraining and cross-validation
Amari, S., Murata, N., Muller, K.-R., Finke, M., & Yang, H. H. (1997) · IEEE Transactions on Neural Networks · doi:10.1109/72.623200 · Cited by 351
Learning and Evolution in Neural Networks
Nolfi, S., Parisi, D., & Elman, J. L. (1994) · Adaptive Behavior · doi:10.1177/105971239400300102 · Cited by 345
The Ellipsoid Method: A Survey
Bland, R. G., Goldfarb, D., & Todd, M. J. (1980) · eCommons (Cornell University) · Cited by 334
A Stochastic Quasi-Newton Method for Online Convex Optimization
Schraudolph, N. N., Yu, J., & Günter, S. (2007) · ANU Open Research (Australian National University) · Cited by 324
Forming Neural Networks Through Efficient and Adaptive Coevolution
Moriarty, D. E., & Miikkulainen, R. (1997) · Evolutionary Computation · doi:10.1162/evco.1997.5.4.373 · Cited by 304
Fogel, D. B., Fogel, L. J., & Porto, V. W. (1990) · Biological Cybernetics · doi:10.1007/bf00199581 · Cited by 299
Tuning the Structure and Parameters of a Neural Network by Using Hybrid Taguchi-Genetic Algorithm
Tsai, J., Chou, J.-H., & Liu, T.-K. (2006) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2005.860885 · Cited by 296
A constructive algorithm for training cooperative neural network ensembles
Islam, M. M., Yao, X., & Murase, K. (2003) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2003.813832 · Cited by 294
Vieira, S. M., Mendonça, L. F., Farinha, G. J., & Sousa, J. M. C. (2013) · Applied Soft Computing · doi:10.1016/j.asoc.2013.03.021 · Cited by 292
Neighborhood based Levenberg-Marquardt algorithm for neural network training
Lera, G., & Pinzolas, M. (2002) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2002.1031951 · Cited by 281
Introduction to Neuro-Fuzzy Systems
Fullér, R. (2000) · Physica-Verlag eBooks · doi:10.1007/978-3-7908-1852-9 · Cited by 276
MCPSO: A multi-swarm cooperative particle swarm optimizer
Niu, B., Zhu, Y., He, X., & Wu, H. (2007) · Applied Mathematics and Computation · doi:10.1016/j.amc.2006.07.026 · Cited by 272
Gene functional classification from heterogeneous data
Pavlidis, P., Weston, J., Cai, J., & Grundy, W. N. (2001) · Proceedings of the fifth annual international conference on Computational biology · doi:10.1145/369133.369228 · Cited by 270
Socha, K., & Blum, C. (2007) · Neural Computing and Applications · doi:10.1007/s00521-007-0084-z · Cited by 265
Sexton, R. S., Dorsey, R. E., & Johnson, J. D. (1998) · Decision Support Systems · doi:10.1016/s0167-9236(97)00040-7 · Cited by 263
Genetic generation of both the weights and architecture for a neural network
Koza, J. R., & Rice, J. P. (2002) · 2022 International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.1991.155366 · Cited by 259
A hybrid approach for feature subset selection using neural networks and ant colony optimization
Sivagaminathan, R. K., & Ramakrishnan, S. (2007) · Expert Systems with Applications · doi:10.1016/j.eswa.2006.04.010 · Cited by 258
Evolutionary artificial neural networks by multi-dimensional particle swarm optimization
Kiranyaz, S., Ince, T., Yildirim, A., & Gabbouj, M. (2009) · Neural Networks · doi:10.1016/j.neunet.2009.05.013 · Cited by 257
Making use of population information in evolutionary artificial neural networks
Yao, X., & Liu, Y. (1998) · IEEE Transactions on Systems, Man and Cybernetics, Part B (Cybernetics) · doi:10.1109/3477.678637 · Cited by 257
An improved PSO-based ANN with simulated annealing technique
Da, Y., & Xiurun, G. (2005) · Neurocomputing · doi:10.1016/j.neucom.2004.07.002 · Cited by 256
A genetic algorithms based multi-objective neural net applied to noisy blast furnace data
Pettersson, F., Chakraborti, N., & Saxén, H. (2007) · Applied Soft Computing · doi:10.1016/j.asoc.2005.09.001 · Cited by 252
Stochastic Modelling and Control
Davis, M. H. A., & Vinter, R. B. (1985) · Springer eBooks · doi:10.1007/978-94-009-4828-0 · Cited by 234
Genetic algorithms and machine learning
Grefenstette, J. J. (1993) · Proceedings of the sixth annual conference on Computational learning theory - COLT '93 · doi:10.1145/168304.168305 · Cited by 228
Ali Ahmadi, M., Zendehboudi, S., Lohi, A., Elkamel, A., & Chatzis, I. (2013) · Geophysical Prospecting · doi:10.1111/j.1365-2478.2012.01080.x · Cited by 227
Complex-Valued Neural Networks
Hirose, A. (2006) · Studies in Computational Intelligence · doi:10.1007/978-3-540-33457-6 · Cited by 226
Khan, K., & Sahai, A. (2012) · International Journal of Intelligent Systems and Applications · doi:10.5815/ijisa.2012.07.03 · Cited by 222
Avoiding false local minima by proper initialization of connections
Wessels, L. F. A., & Barnard, E. (1992) · IEEE Transactions on Neural Networks · doi:10.1109/72.165592 · Cited by 219
COVNET: a cooperative coevolutionary model for evolving artificial neural networks
Garcia-Pedrajas, N., Hervas-Martinez, C., & Munoz-Perez, J. (2003) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2003.810618 · Cited by 211
Stochastic Modelling and Control
Hartley, R. (1986) · Journal of the Operational Research Society · doi:10.1057/jors.1986.158 · Cited by 204
Enhanced MLP performance and fault tolerance resulting from synaptic weight noise during training
Murray, A. F., & Edwards, P. J. (1994) · IEEE Transactions on Neural Networks · doi:10.1109/72.317730 · Cited by 201
Backpropagation-decorrelation: online recurrent learning with O(N) complexity
Steil, J. J. (2005) · 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541) · doi:10.1109/ijcnn.2004.1380039 · Cited by 201
A survey of hybrid ANN/HMM models for automatic speech recognition
Trentin, E., & Gori, M. (2001) · Neurocomputing · doi:10.1016/s0925-2312(00)00308-8 · Cited by 198
The Evolution of Learning: An Experiment in Genetic Connectionism
Chalmers, D. J. (1991) · Connectionist Models · doi:10.1016/b978-1-4832-1448-1.50014-7 · Cited by 193
Lin, C.-J., Chen, C.-H., & Lin, C.-T. (2009) · IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews) · doi:10.1109/tsmcc.2008.2002333 · Cited by 188
Ensemble Learning Using Multi-Objective Evolutionary Algorithms
Chandra, A., & Yao, X. (2006) · Journal of Mathematical Modelling and Algorithms · doi:10.1007/s10852-005-9020-3 · Cited by 179
Quantum artificial neural network architectures and components
Narayanan, A., & Menneer, T. (2000) · Information Sciences · doi:10.1016/s0020-0255(00)00055-4 · Cited by 176
Meta learning evolutionary artificial neural networks
Abraham, A. (2004) · Neurocomputing · doi:10.1016/s0925-2312(03)00369-2 · Cited by 175
Self-citationLocal minima and plateaus in hierarchical structures of multilayer perceptrons
Fukumizu, K., & Amari, S. (2000) · Neural Networks · doi:10.1016/s0893-6080(00)00009-5 · Cited by 172
Using genetic algorithms to select architecture of a feedforward artificial neural network
Arifovic, J., & Gençay, R. (2001) · Physica A: Statistical Mechanics and its Applications · doi:10.1016/s0378-4371(00)00479-9 · Cited by 172
Cited by
Houssein, E. H., Saad, M. R., Hashim, F. A., Shaban, H., & Hassaballah, M. (2020) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2020.103731 · Cited by 499
Kannangara, M., Dua, R., Ahmadi, L., & Bensebaa, F. (2018) · Waste Management · doi:10.1016/j.wasman.2017.11.057 · Cited by 319
Kaveh, M., & Mesgari, M. S. (2023) · Neural Processing Letters · doi:10.1007/s11063-022-11055-6 · Cited by 292
An efficient hybrid multilayer perceptron neural network with grasshopper optimization
Heidari, A. A., Faris, H., Aljarah, I., & Mirjalili, S. (2019) · Soft Computing · doi:10.1007/s00500-018-3424-2 · Cited by 283
A review on the self and dual interactions between machine learning and optimisation
Song, H., Triguero, I., & Özcan, E. (2019) · Progress in Artificial Intelligence · doi:10.1007/s13748-019-00185-z · Cited by 101
Metadata
- Authors
- Ojha, V. K., Abraham, A., & Snášel, V.
- Venue
- Engineering Applications of Artificial Intelligence
- Publisher
- Elsevier BV
- Published
- 2017-04
- Volume
- 60
- Pages
- 97-116
- ISSN
- 0952-1976 (ISSN-L)
- OpenAlex
- W2587345921
- Cited by
- 606
- References
- 301
- FWCI
- 27.06
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.