Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position
The DOI matches its Crossref record; 12 of the 13 references it cites were found in a registry or an index.
Abstract
A neural network model for a mechanism of visual pattern recognition is proposed in this paper. The network is self-organized by "learning without a teacher", and acquires an ability to recognize stimulus patterns based on the geometrical similarity (Gestalt) of their shapes without affected by their positions. This network is given a nickname "neocognitron". After completion of self-organization, the network has a structure similar to the hierarchy model of the visual nervous system proposed by Hubel and Wiesel. The network consists of an input layer (photoreceptor array) followed by a cascade connection of a number of modular structures, each of which is composed of two layers of cells connected in a cascade. The first layer of each module consists of "S-cells", which show characteristics similar to simple cells or lower order hypercomplex cells, and the second layer consists of "C-cells" similar to complex cells or higher order hypercomplex cells. The afferent synapses to each S-cell have plasticity and are modifiable. The network has an ability of unsupervised learning: We do not need any "teacher" during the process of self-organization, and it is only needed to present a set of stimulus patterns repeatedly to the input layer of the network. The network has been simulated on a digital computer. After repetitive presentation of a set of stimulus patterns, each stimulus pattern has become to elicit an output only from one of the C-cells of the last layer, and conversely, t…
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Beyond the single-unit design discussed here, artificial neural networks were inspired by another seemingly universal design principle associated with cortical architecture [255]: the presence of multiple processing layers (such as those found in the visual cortex).”
From Fundamental constraints to the logic of living systems (2024) · cites this paper as [255]
In the citing paper
“Convolutional neural networks (CNNs) constitute one such class of models.9,15,17,19,21,26,32”
From ImageNet classification with deep convolutional neural networks (2017) · cites this paper as [9]
In the citing paper
“In the early 1980s, three new classes of NNs emerged, namely Convolutional Neural Network (CNN) [157], Self-Organizing Map (SOM) [249], and Hopfield network [195].”
From A comprehensive survey on machine learning for networking: Evolution, applications and research opportunities (2018) · cites this paper as [157]
Citations
Cites
Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex
Hubel, D. H., & Wiesel, T. N. (1962) · The Journal of Physiology · doi:10.1113/jphysiol.1962.sp006837 · Cited by 13,680
Ferrier lecture. Functional architecture of macaque monkey visual cortex
Hubel, D. H., & Wiesel, T. N. (1977) · Proceedings of the Royal Society of London. Series B. Biological Sciences · doi:10.1098/rspb.1977.0085 · Cited by 2,608
RECEPTIVE FIELDS AND FUNCTIONAL ARCHITECTURE IN TWO NONSTRIATE VISUAL AREAS (18 AND 19) OF THE CAT
Hubel, D. H., & Wiesel, T. N. (1965) · Journal of Neurophysiology · doi:10.1152/jn.1965.28.2.229 · Cited by 2,603
Mullin, A. A., & Rosenblatt, F. (1963) · The American Mathematical Monthly · doi:10.2307/2312103 · Cited by 2,209
Visual properties of neurons in inferotemporal cortex of the Macaque.
Gross, C. G., Rocha-Miranda, C. E., & Bender, D. B. (1972) · Journal of Neurophysiology · doi:10.1152/jn.1972.35.1.96 · Cited by 1,483 · cited 1×
“), and their responses are scarcely affected by the position or the size of the stimuli (Gross et al., 1972; Sato et al., 1978).”
BackgroundCognitron: A self-organizing multilayered neural network
Fukushima, K. (1975) · Biological Cybernetics · doi:10.1007/bf00342633 · Cited by 757
Self-citationde la Tour, E. B., Bolle, A., & Kellenberger, E. (2008) · Pathologia et Microbiologia · doi:10.1159/000161635 · Cited by 147
A Proposed Model for Visual Information Processing in the Human Brain
Snider, R. S. (1967) · Neurology · doi:10.1212/wnl.17.6.625 · Cited by 52
A Proposed Model for Visual Information Processing in the Human Brain
Kabrisky, M. (1966) · Illinois Digital Environment for Access to Learning and Scholarship (University of Illinois at Urbana-Champaign) · Cited by 44
Feature Extraction and Recognition of Handwritten Characters by Homogeneous Layers
Giebel, H. (1971) · Springer eBooks · doi:10.1007/978-3-642-65175-5_15 · Cited by 13
Research works printed but not linked: 4
- Fukushima, K.: Improvement in pattern-selectivity of a cognitron (in Japanese). Pap. Tech. Group MBE78-27, IECE Japan (1978)
- Fukushima, K.: Self-organization of a neural network which gives position-invariant response. In: Proceedings of the Sixth International Joint Conference on Artificial Intelligence. Tokyo, August 20?23, 1979, pp. 291?293 (1979b)
- Fukushima, K.: Improvement in pattern-selectivity of a cognitron (in Japanese). Trans. IECE Japan (A), J 62-A, 650?657 (1979c)
- Meyer, R.L., Sperry, R.W.: Explanatory models for neuroplasticity in retinotectral connections. In: Plasticity and function in the central nervous system. Stein, D.G., Rosen, J.J., Butters, N. (eds.), pp. 45?63. New York, San Francisco, London: Academic Press 1974
Cited by
Items 701–800 of 1,028
Pixel-Based Machine Learning in Medical Imaging
Suzuki, K. (2012) · International Journal of Biomedical Imaging · doi:10.1155/2012/792079 · Cited by 91
“The convolution NN can be considered as a simplified version of the Neocognitron model [76–78] which was proposed to simulate the human visual system in 1980 [78].”
BackgroundLearning and neural plasticity in visual object recognition
Kourtzi, Z., & DiCarlo, J. J. (2006) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2006.03.012 · Cited by 108
“Computational models [11,43,46] have shown that such explicit object representations can be built using neuronal connections that group together similar features regardless of image changes.”
BackgroundGrape detection with convolutional neural networks
Cecotti, H., Rivera, A., Farhadloo, M., & Pedroza, M. A. (2020) · Expert Systems with Applications · doi:10.1016/j.eswa.2020.113588 · Cited by 103
Sarkar, A., Maniruzzaman, M., Alahe, M. A., & Ahmad, M. (2023) · Journal of Sensors · doi:10.1155/2023/1224619 · Cited by 107
Deep learning of support vector machines with class probability output networks
Kim, S., Yu, Z., Kil, R. M., & Lee, M. (2014) · Neural Networks · doi:10.1016/j.neunet.2014.09.007 · Cited by 97
“…of the importance of deep architecture (Tesauro, 1992; Utgoff & Stracuzzi, 2002), deep learning research has not been prevalent because there has been no useful learning method applicable for existing learning machines except for the models proposed by Fukushima (1980), LeCun et al. (1989).”
MethodDeep Learning Algorithms with Applications to Video Analytics for A Smart City: A Survey
Wang, L., & Sng, D. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1512.03131 · Cited by 78
“In fact, the history of deep learning starts at least from 1980, when Neocognitron [5] is proposed by Fukushima.”
BackgroundA Biologically Inspired Appearance Model for Robust Visual Tracking
Zhang, S., Lan, X., Yao, H., Zhou, H., Tao, D., & Li, X. (2016) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2016.2586194 · Cited by 94
“• Motivation (i): psychophysical evidence has shown that V1 can process visual information using a hierarchical structure [ 37], [38], which alternates between simple and complex cells.”
BackgroundImage Classification Based on the Boost Convolutional Neural Network
Lee, S., Chen, T., Yu, L., & Lai, C. (2018) · IEEE Access · doi:10.1109/access.2018.2796722 · Cited by 103
“Inspired by this, Fukushima and Kunihiko [6] put forward Neocognitron in 1984, which can be regarded as the prototype of themodern convolution network.”
MethodAPPLICATION OF MACHINE AND DEEP LEARNING STRATEGIES FOR THE CLASSIFICATION OF HERITAGE POINT CLOUDS
Grilli, E., Özdemir, E., & Fabio Remondino, A. (2019) · The international archives of the photogrammetry, remote sensing and spatial information sciences/International archives of the photogrammetry, remote sensing and spatial information sciences · doi:10.5194/isprs-archives-xlii-4-w18-447-2019 · Cited by 103
“For the ML approaches and the 1D CNN all the points of the cloud are described by a feature vector that contains different geometric features chosen ad hoc.”
MethodInfluentialJin, Y., Qin, C., Zhang, Z., Tao, J., & Liu, C. (2022) · Science China Technological Sciences · doi:10.1007/s11431-022-2109-4 · Cited by 96
Meng, S., Wu, S., Zhang, J., Cheng, J., Zhou, H., & Zhang, Q. (2024) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2024.3416309 · Cited by 110
“In 1980, [135] proposed a neural network including convolutional and pooling layers.”
BackgroundAbumohsen, M., Owda, A. Y., Owda, M., & Abumihsan, A. (2024) · e-Prime - Advances in Electrical Engineering Electronics and Energy · doi:10.1016/j.prime.2024.100636 · Cited by 90
Advances and Challenges of Single‐Pixel Imaging Based on Deep Learning
Song, K., Bian, Y., Wang, D., Li, R., Wu, K., Liu, H., Qin, C., Hu, J., & Xiao, L. (2024) · Laser & Photonics Review · doi:10.1002/lpor.202401397 · Cited by 103
Unsupervised neural network learning procedures for feature extraction and classification
Becker, S., & Plumbley, M. D. (1996) · Applied Intelligence · doi:10.1007/bf00126625 · Cited by 92
“The Neocognitron has been applied to simple character recognition problems [24].”
BackgroundOvert attention in natural scenes: Objects dominate features
Stoll, J., Thrun, M. C., Nuthmann, A., & Einhäuser, W. (2014) · Vision Research · doi:10.1016/j.visres.2014.11.006 · Cited by 98
Local binary pattern network: A deep learning approach for face recognition
Xi, M., Chen, L., Polajnar, D., & Tong, W. (2016) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2016.7532955 · Cited by 102
Learning to represent visual input
Hinton, G. E. (2010) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2009.0200 · Cited by 91
“A good way to perform this computation in a network of neuron-like processing elements is to use a hierarchy of progressively more complicated feature detectors (Selfridge 1958; Fukushima 1980; LeCun et al. 1998; Serre et al. 2007).”
MethodSantos, J. E., Yin, Y., Jo, H., Pan, W., Kang, Q., Viswanathan, H., Prodanović, M., Pyrcz, M. J., & Lubbers, N. (2021) · Transport in Porous Media · doi:10.1007/s11242-021-01617-y · Cited by 105
A Convolutional Neural Network approach for image-based anomaly detection in smart agriculture
Mendoza-Bernal, J., González-Vidal, A., & Skármeta, A. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.123210 · Cited by 93
Pan, X., Lin, Q. M., Ye, S., Li, L., Guo, L., & Harmon, B. A. (2024) · Heritage Science · doi:10.1186/s40494-024-01179-4 · Cited by 101
“The work [29] compared machine learning based methods, Random Forest, One-versus-One classifier, Convolutional Neural Networks [37] and Recurrent Neural Network [38] to automatically recognizing architectural components such as columns, facades or windows in Temple of Neptune in Paestum and…”
MethodLearning the invariance properties of complex cells from their responses to natural stimuli
Einhäuser, W., Kayser, C., König, P., & Körding, K. P. (2002) · European Journal of Neuroscience · doi:10.1046/j.0953-816x.2001.01885.x · Cited by 91
“Whereas some studies propose a simple linear sum (Fukushima, 1980; Mel, 1997) the max norm has been suggested to ®t physiological ®ndings better when a hierarchy of levels is considered (Riesenhuber & Poggio, 1999).”
BackgroundBuilding and road detection from large aerial imagery
Saito, S., & Aoki, Y. (2015) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.2083273 · Cited by 91
Mapping nonlinear receptive field structure in primate retina at single cone resolution
Freeman, J., Field, G. D., Li, P. H., Greschner, M., Gunning, D. E., Mathieson, K., Sher, A., Litke, A. M., Paninski, L., Simoncelli, E. P., et al. (2015) · eLife · doi:10.7554/elife.05241 · Cited by 90
“, 2013), and may represent a ‘canonical’ computation of sensory processing (Fukushima, 1980; Douglas et al., 1989; Heeger et al., 1996; Riesenhuber and Poggio, 1999).”
BackgroundEmbracing New Techniques in Deep Learning for Estimating Image Memorability
Needell, C. D., & Bainbridge, W. A. (2022) · Computational Brain & Behavior · doi:10.1007/s42113-022-00126-5 · Cited by 109
Automatic land cover classification of geo-tagged field photos by deep learning
Xu, G., Zhu, X., Fu, D., Dong, J., & Xiao, X. (2017) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2017.02.004 · Cited by 90
Kim, H., Jung, W., Park, Y.-C., Lee, J. W., & Ahn, S. (2021) · Expert Systems with Applications · doi:10.1016/j.eswa.2021.116014 · Cited by 90
Nandhini, M., Kala, K. U., Thangadarshini, M., & Verma, S. M. (2022) · Computers and Electronics in Agriculture · doi:10.1016/j.compag.2022.106915 · Cited by 101
Astronomia ex machina: a history, primer and outlook on neural networks in astronomy
Smith, M. J., & Geach, J. E. (2023) · Royal Society Open Science · doi:10.1098/rsos.221454 · Cited by 112
“Sutskever & Hinton’s winning network was a CNN [46] trained through backpropagation [40,93], with ReLU activation [47] and dropout [110] as a regularizer.18 The performance increase afforded by GPU-accelerated training enabled the network to be trained from scratch via backpropagation in a reasonable amount of time.”
BackgroundMethodInfluentialPrinciples for models of neural information processing
Kay, K. N. (2018) · NeuroImage · doi:10.1016/j.neuroimage.2017.08.016 · Cited by 88
“These networks, inspired by properties of biological visual systems (Fukushima, 1980; Serre et al., 2007), consist of multiple layers of processing, where each layer is composed of units that perform relatively simple linear and nonlinear operations on the outputs of previous layers.”
BackgroundCho, K., & Jang, H. (2020) · Scientific Reports · doi:10.1038/s41598-019-56958-y · Cited by 90
Recognition of Indian Sign Language (ISL) Using Deep Learning Model
Sharma, S., & Singh, S. (2021) · Wireless Personal Communications · doi:10.1007/s11277-021-09152-1 · Cited by 100
“CNN is a deep, feed-forward artificial neural network: which is inspired by biological natural visual cognitive mechanism [37], and it can extract millions of translation-invariant features of the underlying object.”
BackgroundVisual Object Recognition: Do We (Finally) Know More Now Than We Did?
Gauthier, L., & Tarr, M. J. (2016) · Annual Review of Vision Science · doi:10.1146/annurev-vision-111815-114621 · Cited by 97
“) Although hierarchical computational models of visual processing have existed for more than 30 years (Fukushima 1980), the scale and speed of computing technology necessarily limited their use.”
BackgroundSegmentation of Bone Structure in X-ray Images using Convolutional Neural Network
CERNAZANU-GLAVAN, C., & HOLBAN, S. (2013) · Advances in Electrical and Computer Engineering · doi:10.4316/aece.2013.01015 · Cited by 86
Unsupervised learning of invariant representations
Anselmi, F., Leibo, J. Z., Rosasco, L., Mutch, J., Tacchetti, A., & Poggio, T. (2016) · Theoretical Computer Science · doi:10.1016/j.tcs.2015.06.048 · Cited by 88
“This is the insight underlying many networks for visual recognition, including HMAX [2] and convolutional neural nets [3] [4].”
BackgroundTHE MORPHOLOGIES OF MASSIVE GALAXIES FROMz∼ 3—WITNESSING THE TWO CHANNELS OF BULGE GROWTH
Pérez‐González, P. G., Mei, S., Shankar, F., Bernardi, M., Daddi, E., Barro, G., Cabrera-Vives, G., Cattaneo, A., Dimauro, P., & Gravet, R. (2015) · The Astrophysical Journal · doi:10.1088/0004-637x/809/1/95 · Cited by 91
“Morphologies are estimated using ConvNets, a specific artificial neural network topology that is inspired by the biological visual cortex (e.g. Fukushima 1980) which is by far the most powerful image classifier up to date.”
MethodHawkes processes for events in social media
Rizoiu, M.-A., Lee, Y., Mishra, S., & Xie, L. (2017) · doi:10.1145/3122865.3122874 · Cited by 128
Stimulus- and goal-oriented frameworks for understanding natural vision
Turner, M. H., Sanchez Giraldo, L. G., Schwartz, O., & Rieke, F. (2019) · Nature Neuroscience · doi:10.1038/s41593-018-0284-0 · Cited by 87
Deep Learning Algorithms for Bearing Fault Diagnostics - A Review
Zhang, S., Zhang, S., Wang, B., & Habetler, T. G. (2019) · IEEE International Symposium on Diagnostics for Electrical Machines, Power Electronics and Drives (SDEMPED) · doi:10.1109/demped.2019.8864915 · Cited by 96
“Inspired by animal visual cortex [19], convolution operation is first introduced to detect image patterns in a hierarchical way from simple to complex features.”
MethodInspecting Method for Defective Casting Products with Convolutional Neural Network (CNN)
Nguyen, T. P., Choi, S.-H., Park, S.-J., Park, S. H., & Yoon, J. (2020) · International Journal of Precision Engineering and Manufacturing-Green Technology · doi:10.1007/s40684-020-00197-4 · Cited by 98
Wen, S., Chen, J., Wu, Y., Yan, Z., Cao, Y., Yang, Y., & Huang, T. (2020) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2020.3019993 · Cited by 92
O’Neill, P. K., Lavrukhin, V., Majumdar, S., Noroozi, V., Zhang, Y., Kuchaiev, O., Balam, J., Dovzhenko, Y., Freyberg, K., Shulman, M. D., et al. (2021) · Interspeech (USB) · doi:10.21437/interspeech.2021-1860 · Cited by 91
The Rise of Machine Learning in Polymer Discovery
Yan, C., & Li, G. (2023) · Advanced Intelligent Systems · doi:10.1002/aisy.202200243 · Cited by 93
A review of hyperspectral image classification based on graph neural networks
Zhao, X., Ma, J., Wang, L., Zhang, Z., Ding, Y., & Xiao, X. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11169-y · Cited by 101
Specialized Representations in Visual Cortex
Ghose, G. M., & Maunsell, J. H. R. (1999) · Neuron · doi:10.1016/s0896-6273(00)80823-5 · Cited by 100
Emotion-modulated attention improves expression recognition: A deep learning model
Barros, P., Parisi, G. I., Weber, C., & Wermter, S. (2017) · Neurocomputing · doi:10.1016/j.neucom.2017.01.096 · Cited by 85
“CNNs were used for several visual recognition tasks, starting ith the Neocognitron proposed by Fukushima [15] .”
MethodAn Industrial Internet Application for Real-Time Fault Diagnosis in Industrial Motors
Langarica, S., Ruffelmacher, C., & Núñez, F. (2019) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2019.2913628 · Cited by 91
“Therefore, vibration-related fault identification is a task in which neural networks can show excellent performance, particularly powerful feature extractors as CNNs. CNNs emerged from the study of the visual cortex in the brain, and they have been used in image recognition since the 1980s [27].”
BackgroundWang, C., Liu, S., Wang, Y., Xiong, J., Zhang, Z., Zhao, B., Luo, L., Lin, G., & He, P. (2022) · Frontiers in Plant Science · doi:10.3389/fpls.2022.868745 · Cited by 104
“A CNN with a convolutional layer and a pooling layer was proposed by Fukushima (1980), which was subsequently improved to LeNet (LeCun et al., 1998), GoogleNet (Szegedy et al., 2015), ResNet (He et al., 2016), AlexNet (Krizhevsky et al., 2017), and so on.”
MethodKha, Q.-H., Ho, Q., & Le, N. Q. K. (2022) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.2c01034 · Cited by 90
Deep Image Features in Music Information Retrieval
Gwardys, G., & Grzywczak, D. (2014) · International Journal of Electronics and Telecommunications · doi:10.2478/eletel-2014-0042 · Cited by 84
“They were introduced by Kunihiko Fukushima in 1980 in [12], [13] and later improved, especially by Y.”
BackgroundA hierarchical neural network model for associative memory
Fukushima, K. (1984) · Biological Cybernetics · doi:10.1007/bf00337157 · Cited by 87
“The hypercolumn of this definition resembles the S-column in the neocognitron (Fukushima, 1980; Fukushima and Miyake, 1982) Ul-1 Ul”
BackgroundMethodSelf-citationMultimodal emotional state recognition using sequence-dependent deep hierarchical features
Barros, P., Jirak, D., Weber, C., & Wermter, S. (2015) · Neural Networks · doi:10.1016/j.neunet.2015.09.009 · Cited by 86
“The Neocognitron, proposed by Fukushima (1980), was the first deep neurocomputational model based on the simple and complex cells, inspiring the CNNs.”
BackgroundWang, L. (1997) · IEEE Transactions on Neural Networks · doi:10.1109/72.623224 · Cited by 89
DualNet: Learn Complementary Features for Image Recognition
Hou, S., Liu, X., & Wang, Z. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.62 · Cited by 85
“The convolutional neural network had its earlier root in [8, 20].”
BackgroundPalani, D., & Venkatalakshmi, K. (2018) · Journal of Medical Systems · doi:10.1007/s10916-018-1139-7 · Cited by 86
Hou, Y., Jia, S., Lun, X., Zhang, S., Chen, T., Wang, F., & Lv, J. (2022) · Frontiers in Bioengineering and Biotechnology · doi:10.3389/fbioe.2021.706229 · Cited by 97
“Apart from RNN, the convolutional neural network (CNN) (Fukushima, 1980; LeCun et al., 1998) has been performed to decode EEG signals as well (Dose et al., 2018; Hou et al., 2020).”
MethodInfluentialApplications of game theory in deep learning: a survey
Hazra, T., & Anjaria, K. (2022) · Multimedia Tools and Applications · doi:10.1007/s11042-022-12153-2 · Cited by 103
Avanzo, M., Stancanello, J., Pirrone, G., Drigo, A., & Retico, A. (2024) · Cancers · doi:10.3390/cancers16213702 · Cited by 106
“…cost varies with earlier weights [80]. enabled the use of neural networks in many hard medical The introduction of the recti fi er function or ReLu ( function, which is zero if the input is lower than the otherwise, helped reduce the risk of vanishing/exploding used activation function as of today.”
BackgroundGao, X. W., James-Reynolds, C., & Currie, E. (2019) · Neurocomputing · doi:10.1016/j.neucom.2018.12.086 · Cited by 94
“Fukushima K, Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position.”
BackgroundData Analytics for Non-Life Insurance PricingPreprint
Wüthrich, M. V., & Buser, C. (2017) · SSRN Electronic Journal · doi:10.2139/ssrn.2870308 · Cited by 134
CFDBench: A Comprehensive Benchmark for Machine Learning Methods in Fluid DynamicsPreprint
Luo, Y., Chen, Y., & Zhang, Z. (2023) · Preprints.org · doi:10.20944/preprints202309.1550.v1 · Cited by 102
Neuronal parts list and wiring diagram for a visual system
Matsliah, A., Yu, S.-C., Kruk, K., Bland, D., Burke, A. T., Gager, J., Hebditch, J., Silverman, B., Willie, K. P., Willie, R., et al. (2024) · Nature · doi:10.1038/s41586-024-07981-1 · Cited by 85
“…discovery of simple and complex cells in the visual cortex not only entered neuroscience textbooks, but the hypothetical neuronal wiring diagrams in their 1962 paper 11 also inspired convolutional nets 12,13 , which eventually ignited the deep-learning revolution in artificial intelligence 14 .”
BackgroundMultigrid Neural Architectures
Ke, T.-W., Maire, M., & Yu, S. X. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.433 · Cited by 103
“Introduction Since Fukushima’s neocognitron [9], the basic architectural design of convolutional neural networks has persisted in form similar to that shown in the top of Figure 1.”
Revealing Detail along the Visual Hierarchy: Neural Clustering Preserves Acuity from V1 to V4
Lu, Y., Yin, J., Chen, Z., Gong, H., Liu, Y., Qian, L., Li, X., Liu, R., Andolina, I. M., & Wang, W. (2018) · Neuron · doi:10.1016/j.neuron.2018.03.009 · Cited by 91
“…filter models of visual processing generate low-level feature invariant responses, for example ignoring position, translation, or object rotation (Fukushima, 1980; LeCun, 1989; Riesenhuber and Poggio, 1999; Yamins and DiCarlo, 2016), but whether invariance is always an optimal strategy for…”
Detection of Flaws in Concrete Using Ultrasonic Tomography and Convolutional Neural Networks
Słoński, M., Schabowicz, K., & Krawczyk, E. (2020) · Materials · doi:10.3390/ma13071557 · Cited by 85
“CNNs go back to the 1980s, but, initially, they were mainly used for optical character recognition (OCR) [26,27].”
BackgroundTemporal pointwise convolutional networks for length of stay prediction in the intensive care unit
Rocheteau, E., Liò, P., & Hyland, S. (2021) · Proceedings of the Conference on Health, Inference, and Learning · doi:10.1145/3450439.3451860 · Cited by 56
“oral trends and inter-feature relationships using a novel combination of techniques. Temporal Convolution Temporal Convolution Networks (TCNs) [26, 55] are a subclass of convolutional neural networks [13] that convolve over the time dimension. They operate on two key principles: the output is the same length as the in…”
MethodAre Deep Neural Networks Adequate Behavioral Models of Human Visual Perception?
Wichmann, F. A., & Geirhos, R. (2023) · Annual Review of Vision Science · doi:10.1146/annurev-vision-120522-031739 · Cited by 70
“(2)As we state, a highly simplified cartoon-history; we are aware of some of the other DNN milestones like the Neocognitron [33], LeNet [75] or HMax [103] which cannot be placed conveniently in the “family history” as presented above.”
BackgroundExplanatory models in neuroscience, Part 1: Taking mechanistic abstraction seriously
Cao, R., & Yamins, D. (2024) · Cognitive Systems Research · doi:10.1016/j.cogsys.2024.101244 · Cited by 64
Feature hierarchies for object classification
Epshtein, B., & Uliman, S. (2005) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2005.98 · Cited by 85
“In a number of these models, [8] [9], the architecture of the hierarchy (size, position and shape of features and their sub-features) is pre-defined rather than learned for different classification tasks.”
BackgroundNonlinear approximation via compositions
Shen, Z., Yang, H., & Zhang, S. (2019) · Neural Networks · doi:10.1016/j.neunet.2019.07.011 · Cited by 124
How Deeply to Fine-Tune a Convolutional Neural Network: A Case Study Using a Histopathology Dataset
Kandel, I., & Castelli, M. (2020) · Applied Sciences · doi:10.3390/app10103359 · Cited by 96
“A 2020 , 10 , 3359 2 of 20 a neural network with more than one hidden layer, it is called deep learning. recent an Deep learning can be implemented in the image classification domain in which a feed-forward convolutional neural network (CNN) [3] can be used to classify images automatically.”
BackgroundMethodMultiple Object Extraction from Aerial Imagery with Convolutional Neural Networks
Saito, S., Yamashita, T., & Aoki, Y. (2015) · Journal of Imaging Science and Technology · doi:10.2352/j.imagingsci.technol.2016.60.1.010402 · Cited by 85
Poggio, T., & Serre, T. (2006) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 106
Simple substrates for complex cognition
Dayan, P. (2008) · Frontiers in Neuroscience · doi:10.3389/neuro.01.031.2008 · Cited by 87
“Conversely, they were too complicated to be readily accommodated by the more straightforward feedforward (Fukushima, 1980; Riesenhuber and Poggio, 2000) or recurrent (Hopfield, 1982) processing architectures popular in supervised, unsupervised, and reinforcement learning (Rumelhart and McClelland,…”
BackgroundA theory of local learning, the learning channel, and the optimality of backpropagation
Baldi, P., & Sadowski, P. (2016) · Neural Networks · doi:10.1016/j.neunet.2016.07.006 · Cited by 74
“While less successful than backpropagation in applications, it has periodically inspired the developmen t of theories aimed at capturing the essence of neural learni ng [26, 22, 29].”
BackgroundDetection of broadleaf weeds growing in turfgrass with convolutional neural networks
Yu, J., Sharpe, S. M., Schumann, A. W., & Boyd, N. S. (2019) · Pest Management Science · doi:10.1002/ps.5349 · Cited by 104
Deep learning and cognitive science
Perconti, P., & Plebe, A. (2020) · Cognition · doi:10.1016/j.cognition.2020.104365 · Cited by 80
Visuo-haptic object perception for robots: an overview
Navarro-Guerrero, N., Toprak, S., Josifovski, J., & Jamone, L. (2023) · Autonomous Robots · doi:10.1007/s10514-023-10091-y · Cited by 105
“This hierarchical organization is what convolutional neural networks (CNNs) take their inspiration from computationally (Fukushima, 1980; LeCun et al., 2015).”
BackgroundImage interpretation by a single bottom-up top-down cycle
Epshtein, B., Lifshitz, I., & Ullman, S. (2008) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0800968105 · Cited by 86
“Hierarchical models of the visual cortex (1–3) typically perform recognition in a feed-forward manner in which recognition proceeds from the detection of simple features to more complex parts to the full object.”
BackgroundMethodThe Architecture of Cognition: Rethinking Fodor and Pylyshyn's Systematicity Challenge
(2014) · The MIT Press eBooks · doi:10.7551/mitpress/9780262027236.001.0001 · Cited by 101
Texture and art with deep neural networks
Gatys, L. A., Ecker, A. S., & Bethge, M. (2017) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2017.08.019 · Cited by 95
Advances in Quantum Deep Learning: An Overview
Garg, S., & Ramakrishnan, G. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.04316 · Cited by 45
“The past decades of deep learning research have led to several breakthroughs such as convolutional neural networks [26, 50, 47] (designed for learning hierarchical and translation-invariant features in images), recurrent neural networks [77, 34] (for sequential data such as time series and natural…”
BackgroundGoing Deeper Into Face Detection: A SurveyPreprint
Minaee, S., Luo, P., Lin, Z., & Bowyer, K. W. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.14983 · Cited by 116
“CNNs were initially proposed by Fukushima [11] in his seminal paper on the “Neocognitron”, which was based on Hubel and Wiesel’s hierarchical receptive field model of the visual cortex.”
MethodA discrete memristive neural network and its application for character recognition
He, S., Liu, J., Wang, H., & Sun, K. (2022) · Neurocomputing · doi:10.1016/j.neucom.2022.12.014 · Cited by 82
The Brain Abstracted: Simplification in the History and Philosophy of Neuroscience
Chirimuuta, M. (2024) · The MIT Press eBooks · doi:10.7551/mitpress/13804.001.0001 · Cited by 132
Singh, M. S., Pondenkandath, V., Zhou, B., Lukowicz, P., & Liwickit, M. (2017) · 2017 International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2017.7966182 · Cited by 76
Performance of Convolutional Neural Networks for Feature Extraction in Froth Flotation Sensing
Horn, Z. C., Auret, L., McCoy, J. T., Aldrich, C., & Herbst, B. M. (2017) · IFAC-PapersOnLine · doi:10.1016/j.ifacol.2017.12.003 · Cited by 82
Short-Term Load Forecasting Based on PSO-KFCM Daily Load Curve Clustering and CNN-LSTM Model
Shang, C., Gao, J., Liu, H., & Liu, F. (2021) · IEEE Access · doi:10.1109/access.2021.3067043 · Cited by 81
“Convolutional neural network (CNN) is a feedforward neural network (FNN) with feature extraction ability, which was proposed by Fukushima [32].”
BackgroundPolarimetric Imaging via Deep Learning: A Review
Li, X., Yan, L., Qi, P., Zhang, L., Goudail, F., Liu, T., Zhai, J., & Hu, H. (2023) · Remote Sensing · doi:10.3390/rs15061540 · Cited by 87
Fine-tuning convolutional deep features for MRI based brain tumor classification
Ahmed, K. B., Hall, L., Goldgof, D. B., Liu, R., & Gatenby, R. A. (2017) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.2253982 · Cited by 81
“They were first introduced in [1], improved in [2] and made larger and deeper in ([3],[4]).”
BackgroundLearning Recognition and Segmentation Using the Cresceptron
Weng, J. J., Ahuja, N., & Huang, T. S. (1997) · International Journal of Computer Vision · doi:10.1023/a:1007967800668 · Cited by 93
Janet, J. P., & Kulik, H. J. (2020) · ACS in focus · doi:10.1021/acs.infocus.7e4001 · Cited by 84
Using deep learning to emulate and accelerate a radiative-transfer model
Lagerquist, R., Turner, D. D., Ebert‐Uphoff, I., Stewart, J. Q., & Hagerty, V. (2021) · Journal of Atmospheric and Oceanic Technology · doi:10.1175/jtech-d-21-0007.1 · Cited by 97
“U-nets are a specialized type of convolutional neural network (CNN; Fukushima 1980; Fukushima and Miyake 1982).”
BackgroundDickinson, Q., & Meyer, J. G. (2022) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1009736 · Cited by 74
“For example, convolutional neural networks (CNNs) are effective at using images as inputs [10], and recurrent neural networks (RNNs) are effective at using sequence data as input [11].”
BackgroundImplanted neural network potentials: Application to Li-Si alloys
Onat, B., Cubuk, E. D., Malone, B. D., & Kaxiras, E. (2018) · Physical Review B · doi:10.1103/physrevb.97.094106 · Cited by 77
Modularity in neural computing
Caelli, T., Guan, L., & Wen, W. (1999) · Proceedings of the IEEE · doi:10.1109/5.784227 · Cited by 86
How deep is the brain? The shallow brain hypothesis
Suzuki, M., Pennartz, C. M. A., & Aru, J. (2023) · Nature Reviews Neuroscience · doi:10.1038/s41583-023-00756-z · Cited by 79
Does Learning Specific Features for Related Parts Help Human Pose Estimation?
Tang, W., & Wu, Y. N. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00120 · Cited by 79
“universally built on convolutional neural networks (CNNs) [11, 21, 20] due to their ability to learn robust feature representations for both images and spatial contexts directly from data.”
BackgroundFinetuning Pretrained Transformers into RNNs
Kasai, J., Peng, H., Zhang, Y., Yogatama, D., Ilharco, G., Pappas, N., Mao, Y., Chen, W., & Smith, N. A. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.830 · Cited by 115
“The relu activation (Fukushima, 1980) ensures that the features are non-negative.4 Different MLP parameters are used for different attention heads, and thus we add a total of rk(d + 1) learnable parameters per layer (less than 0.2% parameter increase in our language model, §3).”
MethodInfluentialJin, Y., Qin, C., Tao, J., & Liu, C. (2021) · Mechanical Systems and Signal Processing · doi:10.1016/j.ymssp.2021.108312 · Cited by 79
Capturing the objects of vision with neural networks
Peters, B., & Kriegeskorte, N. (2021) · Nature Human Behaviour · doi:10.1038/s41562-021-01194-6 · Cited by 75
Metadata
- Authors
- Fukushima, K.
- Affiliations
- Japan Broadcasting Corporation (Japan) (JP)
- Corresponding author
- Fukushima, Kunihiko
- ORCID
- Fukushima, Kunihiko
- Venue
- Biological Cybernetics
- Published
- 1980-04
- Volume
- 36
- Issue
- 4
- Pages
- 193-202
- ISSN
- 0340-1200 (ISSN-L)
- 1432-0770
- OpenAlex
- W613766092
- Semantic Scholar
- 69e68bfaadf2dccff800158749f5a50fe82d173b
- Cited by
- 5,759
- References
- 13
- Citations by year
- 2026: 219 · 2025: 454 · 2024: 494 · 2023: 455 · 2022: 452 · 2021: 544 · 2020: 545 · 2019: 433 · 2018: 301 · 2017: 319 · 2016: 287 · 2015: 176 · 2014: 140 · 2013: 78 · 2012: 71
- FWCI
- 4.02
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.springer.com/tdm (tdm)
- Locations
- Biological Cybernetics (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Life Sciences › Neuroscience › Cognitive Neuroscience › Visual perception and processing mechanisms
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural dynamics and brain function
- Keywords
- neocognitron, visual pattern recognition, unsupervised learning, self-organization, position invariance, S cells, C cells, simple cells, complex cells, hypercomplex cells, visual cortex model, synaptic plasticity, shape recognition
- MeSH terms
- Cognition, Computers, Form Perception, Learning, Models, Neurological, Nerve Net, Nervous System Physiological Phenomena, Pattern Recognition, Visual
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.