Overfitting and neural networks: conjugate gradient and backpropagation
Abstract
Methods for controlling the bias/variance tradeoff typically assume that overfitting or overtraining is a global phenomenon. For multi-layer perceptron (MLP) neural networks, global parameters such as the training time, network size, or the amount of weight decay are commonly used to control the bias/variance tradeoff. However, the degree of overfitting can vary significantly throughout the input space of the model. We show that overselection of the degrees of freedom for an MLP trained with backpropagation can improve the approximation in regions of underfitting, while not significantly overfitting in other regions. This can be a significant advantage over other models. Furthermore, we show that "better" learning algorithms such as conjugate gradient can in fact lead to worse generalization, because they can be more prone to creating varying degrees of overfitting in different regions of the input space. While experimental results can not cover all practical situations, our results do help to explain common behavior that does not agree with theoretical expectations. Our results suggest one important reason for the relative success of MLPs, bring into question common beliefs about neural network training regarding training algorithms, overfitting, and optimal network size, suggest alternate guidelines for practical use, and help to direct future work.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Cascade R-CNN has proven capable of overcoming challenges including overtraining problems and false alarm distribution caused by the patches' proposal stage.
“This framework has proven its capability in overcoming some of the main challenges of object detection, including overtraining problems [5, 6] and false alarm distribution caused by the patches’ proposal stage.”
From A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets (2022) · cites this paper as [5]
Cites
Advances in neural information processing systems 7
(1996) · Computers & Mathematics with Applications · doi:10.1016/s0898-1221(96)90463-0 · Cited by 14,759 · see citing works
Neural Networks and the Bias/Variance Dilemma
Geman, S., Bienenstock, E., & Doursat, R. (1992) · Neural Computation · doi:10.1162/neco.1992.4.1.1 · Cited by 3,578 · see citing works
(1989) · Cited by 3,214 · see citing works
Advances in Neural Information Processing Systems 37
Touretzky, D. S. (2024) · doi:10.52202/079017 · Cited by 2,874 · see citing works
Advances in Neural Information Processing Systems 5
Zemel, R. S., Williams, C. K. I., & Mozer, M. C. (1993) · Cited by 1,896 · see citing works
What Size Net Gives Valid Generalization?
Baum, E. B., & Haussler, D. H. (1989) · Neural Computation · doi:10.1162/neco.1989.1.1.151 · Cited by 1,549 · see citing works
(1992) · Cited by 1,405 · see citing works
Moody, J. E. (1991) · Cited by 437 · see citing works
For Valid Generalization the Size of the Weights is More Important than the Size of the Network
Bartlett, P. L. (1996) · Cited by 213 · see citing works
Note on Learning Rate Schedules for Stochastic Optimization
Darken, C. J., & Moody, J. E. (1990) · neural information processing systems · Cited by 133 · see citing works
Wolpert, D. H. (1997) · Neural Computation · doi:10.1162/neco.1997.9.6.1211 · Cited by 113 · see citing works
Neural networks for speech and sequence recognition
Bengio, Y. (1996) · Medical Entomology and Zoology · Cited by 108 · see citing works
Neural Networks for Speech and Sequence Recognition
Redington, M. (1996) · Language and Speech · doi:10.1177/002383099603900404 · Cited by 72 · see citing works
Flagged first, then by citations
1 of the 14 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Prediction of cardiovascular risk factors from retinal fundus photographs via deep learning
Poplin, R., Varadarajan, A. V., Blumer, K., Liu, Y., McConnell, M. V., Corrado, G. S., Peng, L., & Webster, D. R. (2018) · Nature Biomedical Engineering · doi:10.1038/s41551-018-0195-0 · Cited by 1,865 · see citing works
An analysis of particle swarm optimizers
van den Bergh, F., & Engelbrecht, A. P. (2002) · UpSpace Institutional Repository (University of Pretoria) · Cited by 1,421 · see citing works
Deep learning classifiers for hyperspectral imaging: A review
Paoletti, M. E., Haut, J. M., Plaza, J., & Plaza, A. (2019) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2019.09.006 · Cited by 1,094 · see citing works
A deep learning system for detecting diabetic retinopathy across the disease spectrum
Dai, L., Wu, L., Li, H., Cai, C., Wu, Q., Kong, H., Liu, R., Wang, X., Hou, X., Liu, Y., et al. (2021) · Nature Communications · doi:10.1038/s41467-021-23458-5 · Cited by 620 · see citing works
A systematic review on overfitting control in shallow and deep neural networks
Bejani, M. M., & Ghatee, M. (2021) · Artificial Intelligence Review · doi:10.1007/s10462-021-09975-1 · Cited by 598 · see citing works
Bayoudh, K., Knani, R., Hamdaoui, F., & Mtibaa, A. (2022) · The Visual Computer · doi:10.1007/s00371-021-02166-7 · Cited by 456 · see citing works
Low-N protein engineering with data-efficient deep learning
Biswas, S., Khimulya, G., Alley, E. C., Esvelt, K. M., & Church, G. M. (2021) · Nature Methods · doi:10.1038/s41592-021-01100-y · Cited by 422 · see citing works
Jing, Y., Bian, Y., Hu, Z., Wang, L., & Xie, X.-Q. S. (2018) · The AAPS Journal · doi:10.1208/s12248-018-0210-0 · Cited by 353 · see citing works
Voets, M., Møllersen, K., & Bongo, L. A. (2019) · PLoS ONE · doi:10.1371/journal.pone.0217541 · Cited by 312 · see citing works
Integrated phenology and climate in rice yields prediction using machine learning methods
Guo, Y., Fu, Y. H., Hao, F., Zhang, X., Wu, W., Jin, X., Bryant, C. R., & Senthilnath, J. (2020) · Ecological Indicators · doi:10.1016/j.ecolind.2020.106935 · Cited by 302 · see citing works
Loey, M., Manogaran, G., & Khalifa, N. E. M. (2020) · Neural Computing and Applications · doi:10.1007/s00521-020-05437-x · Cited by 291 · see citing works
Lafdani, E. K., Nia, A. M., & Ahmadi, A. (2012) · Journal of Hydrology · doi:10.1016/j.jhydrol.2012.11.048 · Cited by 239 · see citing works
Yoosefzadeh-Najafabadi, M., Earl, H. J., Tulpan, D., Sulik, J. J., & Eskandari, M. (2021) · Frontiers in Plant Science · doi:10.3389/fpls.2020.624273 · Cited by 226 · see citing works
Prediction of the sequence-specific cleavage activity of Cas9 variants
Kim, N., Kim, H. K., Lee, S., Seo, J. H., Choi, J. W., Park, J., Min, S., Yoon, S., Cho, S.-R., & Kim, H. H. (2020) · Nature Biotechnology · doi:10.1038/s41587-020-0537-9 · Cited by 198 · see citing works
Advances in Artificial Neural Networks – Methodological Development and Application
Huang, Y. (2009) · Algorithms · doi:10.3390/algor2030973 · Cited by 204 · see citing works
Blockchain for deep learning: review and open challenges
Shafay, M., Ahmad, R. W., Salah, K., Yaqoob, I., Jayaraman, R., & Omar, M. (2022) · Cluster Computing · doi:10.1007/s10586-022-03582-7 · Cited by 202 · see citing works
Approaches to the rapid seismic damage prediction of r/c buildings using artificial neural networks
Morfidis, K., & Kostinakis, K. (2018) · Engineering Structures · doi:10.1016/j.engstruct.2018.03.028 · Cited by 192 · see citing works
Montero, A. M. B., Nguyen, D. T. D., Lu, W., Lin, M., Kandalan, R. N., Geets, X., Sterpin, E., & Jiang, S. B. (2019) · Medical Physics · doi:10.1002/mp.13597 · Cited by 192 · see citing works
Deep Learning for Predicting Refractive Error From Retinal Fundus Images
Varadarajan, A. V., Poplin, R., Blumer, K., Angermueller, C., Ledsam, J., Chopra, R., Keane, P. A., Corrado, G. S., Peng, L., & Webster, D. R. (2018) · Investigative Ophthalmology & Visual Science · doi:10.1167/iovs.18-23887 · Cited by 200 · see citing works
Deep Learning for Predicting Refractive Error From Retinal Fundus Images
Varadarajan, A., Poplin, R., Blumer, K., Angermueller, C., Ledsam, J., Chopra, R., Keane, P. A., Corrado, G., Peng, L., & Webster, D. (2018) · UCL Discovery (University College London) · Cited by 174 · see citing works
Deep neural network battery life and voltage prediction by using data of one cycle only
Hsu, C.-W., Xiong, R., Chen, N.-Y., Li, J., & Tsou, N. (2021) · Applied Energy · doi:10.1016/j.apenergy.2021.118134 · Cited by 160 · see citing works
Rebollo, R., Oyoun, F., Corvis, Y., El‐Hammadi, M. M., Saubaméa, B., Andrieux, K., Mignet, N., & Alhareth, K. (2022) · ACS Applied Materials & Interfaces · doi:10.1021/acsami.2c06627 · Cited by 150 · see citing works
A review of critical challenges in MI-BCI: From conventional to deep learning methods
Khademi, Z., Ebrahimi, F., & Kordy, H. M. (2022) · Journal of Neuroscience Methods · doi:10.1016/j.jneumeth.2022.109736 · Cited by 137 · see citing works
Convolutional Neural Networks for Multimedia Sentiment Analysis
Cai, G., & Xia, B. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-25207-0_14 · Cited by 131 · see citing works
Sequence-specific prediction of the efficiencies of adenine and cytosine base editors
Song, M., Kim, H. K., Lee, S., Kim, Y., Seo, S.-Y., Park, J., Choi, J. W., Jang, H., Shin, J. H., Min, S., et al. (2020) · Nature Biotechnology · doi:10.1038/s41587-020-0573-5 · Cited by 124 · see citing works
Arabic aspect based sentiment analysis using bidirectional GRU based models
Abdelgwad, M. M., Soliman, T. H. A., Taloba, A. I., & Farghaly, M. F. (2021) · Journal of King Saud University - Computer and Information Sciences · doi:10.1016/j.jksuci.2021.08.030 · Cited by 132 · see citing works
Overfitting in linear feature extraction for classification of high-dimensional image data
Liu, R., & Gillies, D. F. (2016) · Pattern Recognition · doi:10.1016/j.patcog.2015.11.015 · Cited by 123 · see citing works
Learning to rank with (a lot of) word features
Bai, B., Weston, J., Grangier, D., Collobert, R., Sadamasa, K., Qi, Y., Chapelle, O., & Weinberger, K. Q. (2009) · Information Retrieval · doi:10.1007/s10791-009-9117-9 · Cited by 117 · see citing works
Large neural networks for electricity load forecasting: Are they overfitted?
Hippert, H. S., Bunn, D. W., & Souza, R. C. (2005) · International Journal of Forecasting · doi:10.1016/j.ijforecast.2004.12.004 · Cited by 116 · see citing works
Overfitting in Wrapper-Based Feature Subset Selection: The Harder You Try the Worse it Gets
Loughrey, J., & Cunningham, P. (2007) · Springer eBooks · doi:10.1007/1-84628-102-4_3 · Cited by 109 · see citing works
Artificial neural network applications in the calibration of spark-ignition engines: An overview
Turkson, R. F., Yan, F., Ali, M. K. A., & Hu, J. (2016) · Engineering Science and Technology an International Journal · doi:10.1016/j.jestch.2016.03.003 · Cited by 108 · see citing works
Improving Non-Intrusive Load Disaggregation through an Attention-Based Deep Neural Network
Piccialli, V., & Sudoso, A. M. (2021) · Energies · doi:10.3390/en14040847 · Cited by 107 · see citing works
Zaghloul, M. S., & Achari, G. (2022) · Journal of environmental chemical engineering · doi:10.1016/j.jece.2022.107430 · Cited by 107 · see citing works
Lork, C., Li, W.-T., Qin, Y., Zhou, Y., Yuen, C., Tushar, W., & Saha, T. K. (2020) · Applied Energy · doi:10.1016/j.apenergy.2020.115426 · Cited by 106 · see citing works
Source Code Authorship Attribution Using Long Short-Term Memory Based Networks
Alsulami, B., Dauber, E., Harang, R., Mancoridis, S., & Greenstadt, R. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-66402-6_6 · Cited by 102 · see citing works
Guo, Y., Wang, H., Wu, Z., Wang, S., Sun, H., Senthilnath, J., Wang, J., Bryant, C. R., & Fu, Y. H. (2020) · Sensors · doi:10.3390/s20185055 · Cited by 99 · see citing works
Why Is Optimization Difficult?
Weise, T., Zapf, M., Chiong, R., & Nebro, A. J. (2009) · Studies in computational intelligence · doi:10.1007/978-3-642-00267-0_1 · Cited by 101 · see citing works
Naganna, S. R., Deka, P. C., Ghorbani, M. A., Biazar, S. M., Al-Ansari, N. A., & Yaseen, Z. M. (2019) · Water · doi:10.3390/w11040742 · Cited by 95 · see citing works
Roudsari, A. H., Afshar, J., Lee, W., & Lee, S. (2021) · Scientometrics · doi:10.1007/s11192-021-04179-4 · Cited by 98 · see citing works
A cascaded dual-pathway residual network for lung nodule segmentation in CT images
Liu, H. Q., Cao, H., Song, E., Ma, G., Xu, X., Jin, R., Jin, Y., & Hung, C. (2019) · Physica Medica · doi:10.1016/j.ejmp.2019.06.003 · Cited by 89 · see citing works
Guo, Y., Yin, G., Sun, H., Wang, H., Chen, S., Senthilnath, J., Wang, J., & Fu, Y. H. (2020) · Sensors · doi:10.3390/s20185130 · Cited by 89 · see citing works
Leak diagnosis in pipelines using a combined artificial neural network approach
Pérez-Pérez, E.-J., López‐Estrada, F., Valencia‐Palomo, G., Torres, L., Puig, V., & Antonio, J. D. M. (2020) · Control Engineering Practice · doi:10.1016/j.conengprac.2020.104677 · Cited by 86 · see citing works
A neural network-based predictive model for the thermal conductivity of hybrid nanofluids
Adun, H. H., Wole‐Osho, I., Okonkwo, E. C., Bamisile, O., Dagbasi, M., & Abbasoğlu, S. (2020) · International Communications in Heat and Mass Transfer · doi:10.1016/j.icheatmasstransfer.2020.104930 · Cited by 84 · see citing works
Improving Non-Intrusive Load Disaggregation through an Attention-Based Deep Neural Network
Piccialli, V., & Sudoso, A. M. (2021) · RePEc: Research Papers in Economics · Cited by 83 · see citing works
Deep learning for the fully automated segmentation of the inner ear on MRI
Vaidyanathan, A., van der Lubbe, M. F. J. A., Leijenaar, R. T. H., van Hoof, M., Zerka, F., Miraglio, B., Primakov, S. P., Postma, A. A., Bruintjes, T. D., Bilderbeek-Beckers, M. A. L., et al. (2021) · Scientific Reports · doi:10.1038/s41598-021-82289-y · Cited by 79 · see citing works
Shoulder muscle activation pattern recognition based on sEMG and machine learning algorithms
Jiang, Y., Chen, C., Zhang, X., Chen, C., Zhou, Y. C., Ni, G., Muh, S. J., & Lemos, S. E. (2020) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2020.105721 · Cited by 77 · see citing works
Machine Learning for Silver Nanoparticle Electron Transfer Property Prediction
Sun, B.-C., Fernández, M., & Barnard, A. S. (2017) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.7b00272 · Cited by 62 · see citing works
IoT-Enabled Flood Severity Prediction via Ensemble Machine Learning Models
Khalaf, M., Alaskar, H., Hussain, A. J., Baker, T., Maamar, Z., Buyya, R., Liatsis, P., Khan, W., Tawfik, H., & Al‐Jumeily, D. (2020) · IEEE Access · doi:10.1109/access.2020.2986090 · Cited by 75 · see citing works
A survey on neural networks for (cyber-) security and (cyber-) security of neural networks
Pawlicki, M., Kozik, R., & Choraś, M. (2022) · Neurocomputing · doi:10.1016/j.neucom.2022.06.002 · Cited by 78 · see citing works
Soft computing-based modeling of flotation processes – A review
Jovanović, I., Miljanović, I., & Jovanović, T. (2015) · Minerals Engineering · doi:10.1016/j.mineng.2015.09.020 · Cited by 74 · see citing works
A Novel Approach to Short-Term Stock Price Movement Prediction using Transfer Learning
Nguyen, T.-T., & Yoon, S. (2019) · Applied Sciences · doi:10.3390/app9224745 · Cited by 75 · see citing works
Noia, A. D., Hasekamp, O., van Harten, G., Rietjens, J. H. H., Smit, J. M., Snik, F., Henzing, B., de Boer, J., Keller, C. U., & Volten, H. (2015) · Atmospheric measurement techniques · doi:10.5194/amt-8-281-2015 · Cited by 73 · see citing works
Choi, J., Park, S., & Ahn, J. (2020) · Scientific Reports · doi:10.1038/s41598-020-58821-x · Cited by 72 · see citing works
Ates, G. C., & GÖRGÜLÜARSLAN, R. M. (2021) · Structural and Multidisciplinary Optimization · doi:10.1007/s00158-020-02788-w · Cited by 73 · see citing works
Short-Term Traffic State Prediction Based on the Spatiotemporal Features of Critical Road Sections
Yang, G., Wang, Y., Yu, H., Ren, Y., & Xie, J. (2018) · Sensors · doi:10.3390/s18072287 · Cited by 71 · see citing works
Deep Transfer Learning for Art Classification Problems
Sabatelli, M., Kestemont, M., Daelemans, W. M. P., & Geurts, P. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-11012-3_48 · Cited by 71 · see citing works
Liu, H., Cao, H., Song, E., Ma, G., Xu, X., Jin, R., Liu, C., & Hung, C.-C. (2020) · Journal of Imaging Informatics in Medicine · doi:10.1007/s10278-020-00372-8 · Cited by 71 · see citing works
Development of an ensemble of machine learning algorithms to model aerobic granular sludge reactors
Zaghloul, M. S., Iorhemen, O. T., Hamza, R., Tay, J. H., & Achari, G. (2020) · Water Research · doi:10.1016/j.watres.2020.116657 · Cited by 69 · see citing works
Response selection with topic clues for retrieval-based chatbots
Wu, Y., Li, Z., Wu, W., & Zhou, M. X. (2018) · Neurocomputing · doi:10.1016/j.neucom.2018.07.073 · Cited by 71 · see citing works
Logan, R. A., Williams, B. G., da Silva, M. F., Indani, A., Schcolnicov, N., Ganguly, A., & Miller, S. (2021) · Frontiers in Aging Neuroscience · doi:10.3389/fnagi.2021.720226 · Cited by 67 · see citing works
Revealing hidden patterns in deep neural network feature space continuum via manifold learning
Islam, M. T., Zhou, Z., Ren, H., Khuzani, M. B., Kapp, D. S., Zou, J., Tian, L., Liao, J. C., & Xing, L. (2023) · Nature Communications · doi:10.1038/s41467-023-43958-w · Cited by 65 · see citing works
A sparse Gaussian process regression model for tourism demand forecasting in Hong Kong
Wu, Q. H., Law, R., & Xu, X. (2011) · Expert Systems with Applications · doi:10.1016/j.eswa.2011.09.159 · Cited by 65 · see citing works
Zhao, S., Xu, W., & Chen, L. (2021) · Fuel · doi:10.1016/j.fuel.2021.122966 · Cited by 64 · see citing works
Bai, T., Pang, Y., Wang, J., Han, K., Luo, J., Wang, H., Lin, J., Wu, J., & Zhang, H. (2020) · Remote Sensing · doi:10.3390/rs12050762 · Cited by 64 · see citing works
Furbank, R. T., Silva‐Pérez, V., Evans, J. R., Condon, A. G., Estavillo, G. M., He, W., Newman, S., Poiré, R., Hall, A., & He, Z. (2021) · Plant Methods · doi:10.1186/s13007-021-00806-6 · Cited by 63 · see citing works
Empirical Study of Overfitting in Deep Learning for Predicting Breast Cancer Metastasis
Xu, C., Coen-Pirani, P., & Jiang, X. (2023) · Cancers · doi:10.3390/cancers15071969 · Cited by 63 · see citing works
Maggipinto, M., Terzi, M., Masiero, C., Beghi, A., & Susto, G. A. (2018) · IEEE Transactions on Semiconductor Manufacturing · doi:10.1109/tsm.2018.2849206 · Cited by 63 · see citing works
Deep Learning Architectures for Accurate Millimeter Wave Positioning in 5G
Gante, J., Falcão, G., & Sousa, L. A. (2019) · Neural Processing Letters · doi:10.1007/s11063-019-10073-1 · Cited by 63 · see citing works
He, H.-D., Lu, J. W. Z., & Xue, Y. (2014) · Building and Environment · doi:10.1016/j.buildenv.2014.04.011 · Cited by 60 · see citing works
Ferri, R., Babiloni, C., Karami, V., Triggiani, A. I., Carducci, F., Noce, G., Lizio, R., Pascarelli, M. T., Soricelli, A., Amenta, F., et al. (2020) · Clinical Neurophysiology · doi:10.1016/j.clinph.2020.09.015 · Cited by 59 · see citing works
Development and Validation of an Automated Radiomic CT Signature for Detecting COVID-19
Guiot, J., Vaidyanathan, A., Deprez, L., Zerka, F., Danthine, D., Frix, A.-N., Thys, M., Henket, M. A., Canivet, G., Mathieu, S., et al. (2020) · Diagnostics · doi:10.3390/diagnostics11010041 · Cited by 58 · see citing works
Tian, H., Wu, D. X., Chen, B., Yuan, H., Yu, H., Lou, X., & Chen, C. (2023) · Food Control · doi:10.1016/j.foodcont.2023.109758 · Cited by 57 · see citing works
Alcalá, S. G. S., Antunes, C. H., & Araújo, R. A. M. (2013) · Neurocomputing · doi:10.1016/j.neucom.2013.05.024 · Cited by 56 · see citing works
Financial Time Series Prediction Using Spiking Neural Networks
Reid, D. C., Hussain, A. J., & Tawfik, H. (2014) · PLoS ONE · doi:10.1371/journal.pone.0103656 · Cited by 56 · see citing works
Ensemble pattern trees for predicting hot metal temperature in blast furnace
Zhang, X., Kano, M., & Matsuzaki, S. (2018) · Computers & Chemical Engineering · doi:10.1016/j.compchemeng.2018.10.022 · Cited by 56 · see citing works
Prediction of jacking force using PSO-BPNN and PSO-SVR algorithm in curved pipe roof
Zhou, H., Huang, S., Zhang, P., Ma, B., Ma, P., & Feng, X. (2023) · Tunnelling and Underground Space Technology · doi:10.1016/j.tust.2023.105159 · Cited by 54 · see citing works
Azedou, A., Amine, A., Kisekka, I., Lahssini, S., Bouziani, Y., & Moukrim, S. (2023) · Ecological Informatics · doi:10.1016/j.ecoinf.2023.102333 · Cited by 53 · see citing works
End-to-end sequence labeling via deep learning for automatic extraction of agricultural regulations
Espejo-García, B., Lopez-Pellicer, F. J., Lacasta, J., Moreno, R. P., & Zarazaga-Soria, F. J. (2019) · Computers and Electronics in Agriculture · doi:10.1016/j.compag.2019.03.027 · Cited by 52 · see citing works
Classification of Dental Radiographs Using Deep Learning
de Oro, J. E. C. G., Chaurasia, A., Feldberg, B., Krois, J., & Schwendicke, F. (2021) · Journal of Clinical Medicine · doi:10.3390/jcm10071496 · Cited by 52 · see citing works
Particle swarm optimization trained neural network for aquifer parameter estimation
Ch, S., & Mathur, S. (2012) · KSCE Journal of Civil Engineering · doi:10.1007/s12205-012-1452-5 · Cited by 52 · see citing works
Yin, G., Sun, S., Yu, D., Li, D., & Zhang, K. (2022) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3490686 · Cited by 55 · see citing works
Zhang, J., Liu, S., Yan, H., Li, T., Mao, R., & Liu, J. (2020) · Physics in Medicine and Biology · doi:10.1088/1361-6560/aba87b · Cited by 51 · see citing works
A Deep Learning Model for Classification of Endoscopic Gastroesophageal Reflux Disease
Wang, C., Chiu, Y.-C., Chen, W., Yang, T., Tsai, M., & Tseng, M. (2021) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph18052428 · Cited by 50 · see citing works
Forecasting of ozone level in time series using MLP model with a novel hybrid training algorithm
Wang, D., & Lu, J. W. Z. (2005) · Atmospheric Environment · doi:10.1016/j.atmosenv.2005.10.042 · Cited by 49 · see citing works
Tuplewise Material Representation Based Machine Learning for Accurate Band Gap Prediction
Na, G. S., Jang, S., Lee, Y., & Chang, H. (2020) · The Journal of Physical Chemistry A · doi:10.1021/acs.jpca.0c07802 · Cited by 47 · see citing works
Parkinson’s disease diagnosis using neural networks: Survey and comprehensive evaluation
Tanveer, M., Rashid, A. H., Kumar, R., & Raman, B. (2022) · Information Processing & Management · doi:10.1016/j.ipm.2022.102909 · Cited by 46 · see citing works
An Experimental Analysis of Deep Learning Architectures for Supervised Speech Enhancement
Nossier, S. A., Wall, J. A., Moniri, M., Glackin, C., & Cannings, N. (2020) · Electronics · doi:10.3390/electronics10010017 · Cited by 45 · see citing works
Ensemble neural network model for detecting thyroid eye disease using external photographs
Karlin, J. N., Gai, L., LaPierre, N., Danesh, K., Farajzadeh, J., Palileo, B. M., Taraszka, K., Zheng, J., Wang, W., Eskin, E., et al. (2022) · British Journal of Ophthalmology · doi:10.1136/bjo-2022-321833 · Cited by 45 · see citing works
AI-based histopathology image analysis reveals a distinct subset of endometrial cancers
Darbandsari, A., Farahani, H., Asadi, M., Wiens, M. O., Cochrane, D. R., Mirabadi, A. K., Jamieson, A., Farnell, D., Ahmadvand, P., Douglas, M., et al. (2024) · Nature Communications · doi:10.1038/s41467-024-49017-2 · Cited by 45 · see citing works
Struye, J., & Latré, S. (2019) · Neurocomputing · doi:10.1016/j.neucom.2018.09.098 · Cited by 45 · see citing works
An Improved Random Forest Model for the Prediction of Dam Displacement
Su, Y. P., Weng, K., Lin, C., & Zheng, Z. (2021) · IEEE Access · doi:10.1109/access.2021.3049578 · Cited by 48 · see citing works
Chaudhuri, K. D., & Alkan, B. (2022) · Applied Intelligence · doi:10.1007/s10489-022-03251-7 · Cited by 45 · see citing works
Melchiorre, C., Abella, E. A. C., van Westen, C. J., & Matteucci, M. (2010) · Computers & Geosciences · doi:10.1016/j.cageo.2010.10.004 · Cited by 44 · see citing works
Application of Ridge Polynomial Neural Networks to Financial Time Series Prediction
Ghazali, R., Hussain, A. J., & El‐Deredy, W. (2006) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2006.1716194 · Cited by 44 · see citing works
Diagnosing acute promyelocytic leukemia by using convolutional neural network
Ouyang, N., Wang, W., Ma, L., Wang, Y., Chen, Q., Yang, S., Xie, J., Su, S., Cheng, Y., Cheng, Q., et al. (2020) · Clinica Chimica Acta · doi:10.1016/j.cca.2020.10.039 · Cited by 44 · see citing works
Remarks on the Use of Multilayer Perceptrons for the Analysis of Chemical Sensor Array Data
Pardo, M., & Sberveglieri, G. (2004) · IEEE Sensors Journal · doi:10.1109/jsen.2004.827207 · Cited by 44 · see citing works
A New Early Stopping Algorithm for Improving Neural Network Generalization
Wu, X., & Liu, J. (2009) · International Conference on Intelligent Computation Technology and Automation · doi:10.1109/icicta.2009.11 · Cited by 43 · see citing works
A neural network approach to understanding implied volatility movements
Cao, J., Chen, J. H. K., & Hull, J. C. (2020) · Quantitative Finance · doi:10.1080/14697688.2020.1750679 · Cited by 43 · see citing works
Li, S., & Sun, L. (2020) · Journal of Bridge Engineering · doi:10.1061/(asce)be.1943-5592.0001531 · Cited by 43 · see citing works
Remote Sensing Scene Classification and Explanation Using RSSCNet and LIME
Hung, S.-C., Wu, H.-C., & Tseng, M. (2020) · Applied Sciences · doi:10.3390/app10186151 · Cited by 43 · see citing works
- Authors
- Lawrence, S., & Giles, C. L.
- Affiliations
- NEC (Japan) (JP)
- ORCID
- Giles, C.L.
- Venue
- Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks. IJCNN 2000. Neural Computing: New Challenges and Perspectives for the New Millennium
- Publisher
- IEEE
- Published
- 2000
- Pages
- 114-119 vol.1
- ISSN
- 2161-4407 (ISSN-L)
- OpenAlex
- W2114367267
- Cited by
- 491
- References
- 14
- Citations by year
- 2026: 11 · 2025: 32 · 2024: 41 · 2023: 42 · 2022: 51 · 2021: 67 · 2020: 60 · 2019: 44 · 2018: 23 · 2017: 14 · 2016: 6 · 2015: 8 · 2014: 10 · 2013: 10 · 2012: 8
- FWCI
- 2.54
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- International Joint Conference on Neural Networks (IJCNN) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and ELM
- Keywords
- overfitting, bias-variance trade-off, multilayer perceptron, conjugate gradient method, backpropagation, generalization, learning algorithms, network size, weight decay, input space
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review