Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position
The DOI matches its Crossref record; 12 of the 13 references it cites were found in a registry or an index.
Abstract
A neural network model for a mechanism of visual pattern recognition is proposed in this paper. The network is self-organized by "learning without a teacher", and acquires an ability to recognize stimulus patterns based on the geometrical similarity (Gestalt) of their shapes without affected by their positions. This network is given a nickname "neocognitron". After completion of self-organization, the network has a structure similar to the hierarchy model of the visual nervous system proposed by Hubel and Wiesel. The network consists of an input layer (photoreceptor array) followed by a cascade connection of a number of modular structures, each of which is composed of two layers of cells connected in a cascade. The first layer of each module consists of "S-cells", which show characteristics similar to simple cells or lower order hypercomplex cells, and the second layer consists of "C-cells" similar to complex cells or higher order hypercomplex cells. The afferent synapses to each S-cell have plasticity and are modifiable. The network has an ability of unsupervised learning: We do not need any "teacher" during the process of self-organization, and it is only needed to present a set of stimulus patterns repeatedly to the input layer of the network. The network has been simulated on a digital computer. After repetitive presentation of a set of stimulus patterns, each stimulus pattern has become to elicit an output only from one of the C-cells of the last layer, and conversely, t…
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Beyond the single-unit design discussed here, artificial neural networks were inspired by another seemingly universal design principle associated with cortical architecture [255]: the presence of multiple processing layers (such as those found in the visual cortex).”
From Fundamental constraints to the logic of living systems (2024) · cites this paper as [255]
In the citing paper
“Convolutional neural networks (CNNs) constitute one such class of models.9,15,17,19,21,26,32”
From ImageNet classification with deep convolutional neural networks (2017) · cites this paper as [9]
In the citing paper
“In the early 1980s, three new classes of NNs emerged, namely Convolutional Neural Network (CNN) [157], Self-Organizing Map (SOM) [249], and Hopfield network [195].”
From A comprehensive survey on machine learning for networking: Evolution, applications and research opportunities (2018) · cites this paper as [157]
Citations
Cites
Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex
Hubel, D. H., & Wiesel, T. N. (1962) · The Journal of Physiology · doi:10.1113/jphysiol.1962.sp006837 · Cited by 13,680
Ferrier lecture. Functional architecture of macaque monkey visual cortex
Hubel, D. H., & Wiesel, T. N. (1977) · Proceedings of the Royal Society of London. Series B. Biological Sciences · doi:10.1098/rspb.1977.0085 · Cited by 2,608
RECEPTIVE FIELDS AND FUNCTIONAL ARCHITECTURE IN TWO NONSTRIATE VISUAL AREAS (18 AND 19) OF THE CAT
Hubel, D. H., & Wiesel, T. N. (1965) · Journal of Neurophysiology · doi:10.1152/jn.1965.28.2.229 · Cited by 2,603
Mullin, A. A., & Rosenblatt, F. (1963) · The American Mathematical Monthly · doi:10.2307/2312103 · Cited by 2,209
Visual properties of neurons in inferotemporal cortex of the Macaque.
Gross, C. G., Rocha-Miranda, C. E., & Bender, D. B. (1972) · Journal of Neurophysiology · doi:10.1152/jn.1972.35.1.96 · Cited by 1,483 · cited 1×
“), and their responses are scarcely affected by the position or the size of the stimuli (Gross et al., 1972; Sato et al., 1978).”
BackgroundCognitron: A self-organizing multilayered neural network
Fukushima, K. (1975) · Biological Cybernetics · doi:10.1007/bf00342633 · Cited by 757
Self-citationde la Tour, E. B., Bolle, A., & Kellenberger, E. (2008) · Pathologia et Microbiologia · doi:10.1159/000161635 · Cited by 147
A Proposed Model for Visual Information Processing in the Human Brain
Snider, R. S. (1967) · Neurology · doi:10.1212/wnl.17.6.625 · Cited by 52
A Proposed Model for Visual Information Processing in the Human Brain
Kabrisky, M. (1966) · Illinois Digital Environment for Access to Learning and Scholarship (University of Illinois at Urbana-Champaign) · Cited by 44
Feature Extraction and Recognition of Handwritten Characters by Homogeneous Layers
Giebel, H. (1971) · Springer eBooks · doi:10.1007/978-3-642-65175-5_15 · Cited by 13
Research works printed but not linked: 4
- Fukushima, K.: Improvement in pattern-selectivity of a cognitron (in Japanese). Pap. Tech. Group MBE78-27, IECE Japan (1978)
- Fukushima, K.: Self-organization of a neural network which gives position-invariant response. In: Proceedings of the Sixth International Joint Conference on Artificial Intelligence. Tokyo, August 20?23, 1979, pp. 291?293 (1979b)
- Fukushima, K.: Improvement in pattern-selectivity of a cognitron (in Japanese). Trans. IECE Japan (A), J 62-A, 650?657 (1979c)
- Meyer, R.L., Sperry, R.W.: Explanatory models for neuroplasticity in retinotectral connections. In: Plasticity and function in the central nervous system. Stein, D.G., Rosen, J.J., Butters, N. (eds.), pp. 45?63. New York, San Francisco, London: Academic Press 1974
Cited by
Items 101–200 of 1,028
Machine learning for internet of things data analysis: a survey
Mahdavinejad, M. S., Rezvan, M., Barekatain, M., Adibi, P., Barnaghi, P., & Sheth, A. P. (2018) · Digital Communications and Networks · doi:10.1016/j.dcan.2017.10.002 · Cited by 921
“With enough hidden units, an FFNN with at least two layers can approximate an arbitrary mapping from a fi nite input space to a fi nite output space [121 – 123].”
MethodElements of Artificial Neural Networks
Mehrotra, K., Mohan, C., & Ranka, S. (1996) · The MIT Press eBooks · doi:10.7551/mitpress/2687.001.0001 · Cited by 912
A feedforward architecture accounts for rapid categorization
Serre, T., Oliva, A., & Poggio, T. (2007) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0700622104 · Cited by 918
Ghamisi, P., Yokoya, N., Li, J., Liao, W., Liu, S., Plaza, J., Rasti, B., & Plaza, A. (2017) · IEEE Geoscience and Remote Sensing Magazine · doi:10.1109/mgrs.2017.2762087 · Cited by 909
“, stacked autoencoders [62] and convolutional neural networks (CNNs) [63], are used for spectral-spatial feature extraction for HSI classification [64], [65].”
MethodOlshausen, B., Anderson, C., & Van Essen, D. (1993) · The Journal of Neuroscience · doi:10.1523/jneurosci.13-11-04700.1993 · Cited by 908
Segmenting Retinal Blood Vessels WithDeep Neural Networks
Liskowski, P., & Krawiec, K. (2016) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2016.2546227 · Cited by 890
“It is also consistent with the spatial nature of visual information and mimics certain aspects of natural visual systems [1].”
BackgroundDeep Learning for Time Series Forecasting: A Survey
Torres, J. F., Hadjout, D., Sebaa, A., Martínez-Álvarez, F., & Troncoso, A. (2021) · Big Data · doi:10.1089/big.2020.0159 · Cited by 818
“36 by Fukushima and are one of the most common architectures in image processing and computer vision.”
Deep learning in fluid dynamics
Kutz, J. N. (2017) · Journal of Fluid Mechanics · doi:10.1017/jfm.2016.803 · Cited by 831
“The first mathematical model of a neural network, termed the Neocognitron in 1980 (Fukushima 1980), had many of the characteristic features of today’s deep neural networks (DNNs), which are typically between 7–10 layers, but more recently have been scaled to hundreds of layers for certain…”
BackgroundObject Recognition with Features Inspired by Visual Cortex
Serre, T., Wolf, L., & Poggio, T. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.254 · Cited by 821
“Models of biological vision [5, 13, 16, 1] have not been extended to deal with real-world object recognition tasks (e.”
BackgroundSTDP-based spiking deep convolutional neural networks for object recognition
Kheradpisheh, S. R., Ganjtabesh, M., Thorpe, S. J., & Masquelier, T. (2018) · Neural Networks · doi:10.1016/j.neunet.2017.12.005 · Cited by 768
“During the last decades, various computational models have been proposed to mimic this hierarchical feed-forward processing [15, 28, 48, 36, 31].”
BackgroundCadieu, C. F., Hong, H., Yamins, D. L. K., Pinto, N., Ardila, D., Solomon, E. A., Majaj, N. J., & DiCarlo, J. J. (2014) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1003963 · Cited by 824
“Models of biological vision have extended this hypothesis by suggesting that higher visual areas recapitulate this mechanism and form a hierarchy [12,13,19–22].”
BackgroundConvolutional networks for fast, energy-efficient neuromorphic computing
Esser, S. K., Merolla, P. A., Arthur, J. V., Cassidy, A. S., Appuswamy, R., Andreopoulos, A., Berg, D. J., McKinstry, J. L., Melano, T., Barch, D. R., et al. (2016) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1604850113 · Cited by 781
“On the algorithm front, building on classic work on backpropagation (1), the neocognitron (2), and convolutional networks (3), deep learning has made great strides in achieving human-level performance on a wide range of recognition tasks (4).”
BackgroundInfluentialEmotion recognition using multi-modal data and machine learning techniques: A tutorial and review
Zhang, J., Yin, Z., Chen, P., & Nichele, S. (2020) · Information Fusion · doi:10.1016/j.inffus.2020.01.011 · Cited by 806
Human Pose Estimation with Iterative Error Feedback
Carreira, J., Agrawal, P., Fragkiadaki, K., & Malik, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.512 · Cited by 805
“Feature extractors such as Convolutional Networks (ConvNets) [23] represent images using a multi-layered hierarchy of features and are inspired by the structure and functionality of the visual pathway of the human brain [13, 1].”
BackgroundArtificial intelligence in the creative industries: a review
Anantrasirichai, N., & Bull, D. (2022) · Artificial Intelligence Review · doi:10.1007/s10462-021-10039-7 · Cited by 793
“The first CNN was designed by Fukushima (1980) as a tool for visual pattern recognition (Fig.”
MethodAcharya, U. R., Fujita, H., Lih, O. S., Hagiwara, Y., Tan, J. H., & Adam, M. (2017) · Information Sciences · doi:10.1016/j.ins.2017.04.012 · Cited by 752
Rotation-invariant convolutional neural networks for galaxy morphology prediction
Dieleman, S., Willett, K. W., & Dambre, J. (2015) · Monthly Notices of the Royal Astronomical Society · doi:10.1093/mnras/stv632 · Cited by 781
“Although neural networks have existed for several decades (McCulloch & Pitts 1943; Fukushima 1980), they have recently returned to the forefront of machine learning research.”
BackgroundDeep learning in optical metrology: a review
Zuo, C., Qian, J., Feng, S., Yin, W., Li, Y., Fan, P., Han, J., Qian, K., & Chen, Q. (2022) · Light: Science & Applications · doi:10.1038/s41377-022-00714-x · Cited by 711
Fukushima, K., & Miyake, S. (1982) · Pattern Recognition · doi:10.1016/0031-3203(82)90024-3 · Cited by 742
Self-citationWhat are the computations of the cerebellum, the basal ganglia and the cerebral cortex?
Doya, K. (1999) · Neural Networks · doi:10.1016/s0893-6080(99)00046-5 · Cited by 747
Matsugu, M., Mori, K., Mitari, Y., & Kaneda, Y. (2003) · Neural Networks · doi:10.1016/s0893-6080(03)00115-1 · Cited by 715
“Convolutional neural network (CNN) models (Le Cun & Bengio, 1995) as well as neocognitrons (Fukushima, 1980) known as one of biologically inspired models, have been used for pattern recognition tasks such as face recognition and hand-written numeral recognition.”
BackgroundA Biologically Inspired System for Action Recognition
Jhuang, H., Serre, T., Wolf, L., & Poggio, T. (2007) · 2007 IEEE 11th International Conference on Computer Vision · doi:10.1109/iccv.2007.4408988 · Cited by 723
“These systems have been around for quiet some time now, starting with the work of Fukushima [8] and LeCun et al.”
MethodJo, T., Nho, K., & Saykin, A. J. (2019) · Frontiers in Aging Neuroscience · doi:10.3389/fnagi.2019.00220 · Cited by 724
“…processing units called artificial neurons that were modeled (McCulloch and Pitts, 1943) and developed with the concept of Perceptron (Rosenblatt, 1957;1958), Group Method of Data Handling (GMDH) (Ivakhnenko and Lapa, 1965;Ivakhnenko, 1968;1971) and the Neocognitron (Fukushima, 1979;1980).”
MethodA review on deep convolutional neural networks
Aloysius, N., & Geetha, M. (2017) · 2017 International Conference on Communication and Signal Processing (ICCSP) · doi:10.1109/iccsp.2017.8286426 · Cited by 636
“The neocognitron in 1980 [10] is considered as the predecessor of ConvNets.”
BackgroundGeodesic Convolutional Neural Networks on Riemannian Manifolds
Masci, J., Boscaini, D., Bronstein, M. M., & Vandergheynst, P. (2015) · 2015 IEEE International Conference on Computer Vision Workshop (ICCVW) · doi:10.1109/iccvw.2015.112 · Cited by 721
“The past decade in computer vision research has witnessed the re-emergence of " deep learning " and in particular, convolutional neural network (CNN) techniques [17] [27], allowing to learn taskspecific features from examples.”
BackgroundResidual Networks Behave Like Ensembles of Relatively Shallow Networks
Veit, A., Wilber, M., & Belongie, S. (2016) · arXiv · doi:10.48550/arxiv.1605.06431 · Cited by 705
“This organization has been widely adopted in the computer vision and machine learning literature, from early neural networks such as the Neocognitron [4] and the traditional hand-crafted feature pipeline of Malik and Perona [15] to convolutional neural networks [14, 13].”
MethodChallenges in representation learning: A report on three machine learning contests
Goodfellow, I. J., Erhan, D., Luc Carrier, P., Courville, A., Mirza, M., Hamner, B., Cukierski, W., Tang, Y., Thaler, D., Lee, D.-H., et al. (2015) · Neural Networks · doi:10.1016/j.neunet.2014.09.005 · Cited by 697
Breast Cancer Multi-classification from Histopathological Images with Structured Deep Learning Model
Han, Z., Wei, B., Zheng, Y., Yin, Y., Li, K., & Li, S. (2017) · Scientific Reports · doi:10.1038/s41598-017-04075-z · Cited by 683
“Furthermore, although CNNs inspired by Kunihiko Fukushima 26, 27 , has been used for medical image analysis, e.g., image segmentation 28, 29 image fusion and registration 30–32 , but there still exists a lot of room for improvement of medical data in comparison with the computer vision domain 7,…”
MethodRiesenhuber, M., & Poggio, T. (2000) · Nature Neuroscience · doi:10.1038/81479 · Cited by 658
Deep Learning for Plant Stress Phenotyping: Trends and Future Perspectives
Singh, A. K., Ganapathysubramanian, B., Sarkar, S., & Singh, A. (2018) · Trends in Plant Science · doi:10.1016/j.tplants.2018.07.004 · Cited by 635
Metamers of the ventral stream
Freeman, J., & Simoncelli, E. P. (2011) · Nature Neuroscience · doi:10.1038/nn.2889 · Cited by 667
“In each block of the experiment, observers performed 288 trials, one for each combination of image (4), scaling (6), and trial type (12).”
MethodLearning invariance from transformation sequences
Földiák, P. (1991) · Neural Computation · doi:10.1162/neco.1991.3.2.194 · Cited by 661
Learning Temporal Information for Brain-Computer Interface Using Convolutional Neural Networks
Sakhavi, S., Guan, C., & Yan, S. (2018) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2018.2789927 · Cited by 656
“The concept of hierarchical convolutional data processing (used in CNN) was initially introduced by Fukushima in the neo-cognition framework [3] in 1980 and further developed by LeCun in the LeNet-5 architecture [4] in 1998.”
MethodMachine learning and its applications to biology
Tarca, A. L., Carey, V. J., Chen, X.-W., Romero, R., & Drăghici, S. (2007) · PLoS Computational Biology · doi:10.1371/journal.pcbi.0030116 · Cited by 656
“Further artificial neural network architectures such as the adaptive resonance theory (ART) [3] and neocognitron [4] were inspired from the organization of the visual nervous system.”
BackgroundAn Introductory Review of Deep Learning for Prediction Models With Big Data
Emmert-Streib, F., Yang, Z., Feng, H., Tripathi, S., & Dehmer, M. (2020) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2020.00004 · Cited by 604
“In 1980 , a hierarchical multilayered neural network for visual pattern recognition called Neocognitron was introduced by Fukushima (1980).”
BackgroundDeep Learning and Its Applications in Biomedicine
Cao, C., Liu, F., Tan, H., Song, D., Shu, W., Li, W., Zhou, Y., Bo, X., & Xie, Z. (2018) · Genomics, Proteomics & Bioinformatics · doi:10.1016/j.gpb.2017.07.003 · Cited by 624
“The early framework for deep learning was built on artificial neural networks (ANNs) in the 1980s [2], while the real impact of deep learning became apparent in 2006 [3,4].”
BackgroundLocal Relation Networks for Image Recognition
Hu, H., Zhang, Z., Xie, Z., & Lin, S. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00356 · Cited by 570
“Convolution Layers and Extensions The convolution layer has existed for several decades [8, 18].”
BackgroundSurvey of spiking in the mouse visual system reveals functional hierarchy
Siegle, J. H., Jia, X., Durand, S., Gale, S., Bennett, C., Graddis, N., Heller, G., Ramirez, T. K., Choi, H., Luviano, J. A., et al. (2021) · Nature · doi:10.1038/s41586-020-03171-x · Cited by 634
View Synthesis by Appearance Flow
Zhou, T., Tulsiani, S., Sun, W., Malik, J., & Efros, A. A. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_18 · Cited by 579
“A recent interest in learning to synthesize views for more challenging objects under diverse view variations has been driven by the ability of Convolutional Neural Networks (CNNs) [15,16] to function as image decoders.”
BackgroundVehicle Detection in Satellite Images by Hybrid Deep Convolutional Neural Networks
Chen, X., Xiang, S., Liu, C.-L., & Pan, C.-H. (2014) · IEEE Geoscience and Remote Sensing Letters · doi:10.1109/lgrs.2014.2309695 · Cited by 622
“The max-pooling layer was first introduced by Fukushima [12].”
BackgroundAdvanced Deep-Learning Techniques for Salient and Category-Specific Object Detection: A Survey
Han, J., Zhang, D., Cheng, G., Liu, N., & Xu, D. (2018) · IEEE Signal Processing Magazine · doi:10.1109/msp.2017.2749125 · Cited by 621
“The CNN is one of the most well-known and widely used deep-learning architectures inspired by the natural visual perception mechanism of living creatures, which was first proposed in 1980 by Fukushima [21] and then improved by LeCun [22].”
BackgroundEvaluation of convolutional neural networks for visual recognition
Nebauer, C. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/72.701181 · Cited by 597
Cnvlutin: ineffectual-neuron-free deep neural network computing
Albericio, J., Judd, P., Hetherington, T., Aamodt, T., Jerger, N. E., & Moshovos, A. (2016) · ACM SIGARCH Computer Architecture News · doi:10.1145/3007787.3001138 · Cited by 614
“DNN are not new [1], but are currently enjoying a renaissance [2] in part due to the increase in computing capabilities available in commodity computing platforms such as general purpose graphics processors [3].”
Constrained Convolutional Neural Networks for Weakly Supervised Segmentation
Pathak, D., Krahenbuhl, P., & Darrell, T. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.209 · Cited by 617
Project Adam: building an efficient and scalable deep learning training system
Chilimbi, T. M., Suzue, Y., Apacible, J., & Kalyanaraman, K. (2014) · Operating Systems Design and Implementation · doi:10.5555/2685048.2685094 · Cited by 550
A Neural Theory of Visual Attention: Bridging Cognition and Neurophysiology.
Bundesen, C., Habekost, T., & Kyllingsbæk, S. (2005) · Psychological Review · doi:10.1037/0033-295x.112.2.291 · Cited by 585
“…of position and size invariance in visual recognition (for different approaches, see Heinke & Humphreys, 2003; Riesenhuber & Poggio, 2000; see also Fukushima, 1980; Larsen & Bundesen, 1978, 1998), we assume (a) that the effective RF can vary in position (shifting) and size (scaling) within the…”
BackgroundConvolutional Neural Networks as a Model of the Visual System: Past, Present, and Future
Lindsay, G. W. (2021) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_01544 · Cited by 557
“In 1980, Fukushima transformed Hubel and Wiesel’s findings into a functioning model of the visual system (Fukushima, 1980).”
BackgroundMachine learning in acoustics: Theory and applications
Bianco, M. J., Gerstoft, P., Traer, J., Ozanich, E., Roch, M. A., Gannot, S., & Deledalle, C.-A. (2019) · The Journal of the Acoustical Society of America · doi:10.1121/1.5133944 · Cited by 609
“Convolutional NNs (CNNs) 77,78 are an alternative to conventional, fully connected NNs for temporally or spatially correlated signals.”
BackgroundMethodGupta, A., Anpalagan, A., Guan, L., & Khwaja, A. S. (2021) · Array · doi:10.1016/j.array.2021.100057 · Cited by 565
Shadlen, M. N., & Movshon, J. A. (1999) · Neuron · doi:10.1016/s0896-6273(00)80822-3 · Cited by 606
Weed detection in soybean crops using ConvNets
dos Santos Ferreira, A., Matte Freitas, D., Gonçalves da Silva, G., Pistori, H., & Theophilo Folhes, M. (2017) · Computers and Electronics in Agriculture · doi:10.1016/j.compag.2017.10.027 · Cited by 581
“The first computational models based on these local connectivities between neurons and on hierarchically organized image transformations are found in Fukushima’s Neocognitron (Fukushima, 1980).”
BackgroundA time-delay neural network architecture for isolated word recognition
Lang, K. J., Waibel, A. H., & Hinton, G. E. (1990) · Neural Networks · doi:10.1016/0893-6080(90)90044-l · Cited by 596
A critique of pure learning and what artificial neural networks can learn from animal brains
Zador, A. M. (2019) · Nature Communications · doi:10.1038/s41467-019-11786-6 · Cited by 536
Early Convolutions Help Transformers See Better
Xiao, T., Singh, M., Mintun, E., Darrell, T. J., Dollár, P., & Girshick, R. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.14881 · Cited by 339
“The breakthrough performance of the AlexNet [23] CNN [15, 24] on ImageNet classification [10] transformed the field of recognition, leading to the development of higher performing architectures, e.”
MethodApplications of machine learning to diagnosis and treatment of neurodegenerative diseases
Myszczynska, M. A., Ojamies, P. N., Lacoste, A. M. B., Neil, D., Saffari, A., Mead, R., Hautbergue, G. M., Holbrook, J. D., & Ferraiuolo, L. (2020) · Nature Reviews Neurology · doi:10.1038/s41582-020-0377-8 · Cited by 595
Land Use Classification in Remote Sensing Images by Convolutional Neural Networks
Castelluccio, M., Poggi, G., Sansone, C., & Verdoliva, L. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1508.00092 · Cited by 474
“These problems are mostly solved by CNNs. Convolutional neural networks were first proposed in 1980 by Fukushima [44] (called NeoCognitron) and then refined by LeCun [45].”
MethodMulti-digit Number Recognition from Street View Imagery using Deep Convolutional Neural Networks
Goodfellow, I. J., Bulatov, Y., Ibarz, J., Arnoud, S., & Shet, V. (2013) · arXiv · doi:10.48550/arxiv.1312.6082 · Cited by 582
“Convolutional neural networks (Fukushima, 1980; LeCun et al. , 1998) are neural networks with sets of neurons having tied parameters.”
BackgroundUnsupervised Learning of Visual Features through Spike Timing Dependent Plasticity
Masquelier, T., & Thorpe, S. J. (2007) · PLoS Computational Biology · doi:10.1371/journal.pcbi.0030031 · Cited by 548
“The third main simplification we have made consists of using restricted receptive fields and weight sharing, as do most of the bio-inspired hierarchical networks [6–10] (networks using these techniques are called convolutional networks).”
BackgroundMethodWhy and when can deep-but not shallow-networks avoid the curse of dimensionality: A review
Poggio, T., Mhaskar, H., Rosasco, L., Miranda, B., & Liao, Q. (2017) · International Journal of Automation and Computing · doi:10.1007/s11633-017-1054-2 · Cited by 559
“The Neocognitron([8]) was a convolutional neural network that was trained to recognize characters.”
BackgroundDeepContour: A deep convolutional feature learned by positive-sharing loss for contour detection
Shen, W., Wang, X., Wang, Y., Bai, X., & Zhang, Z. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299024 · Cited by 515
“However, we do not just use the CNN as a blackbox feature extractor but derive it by considering the intrinsic properties of contours.”
MethodComputational Neuroscience of Vision
Rolls, E., & Deco, G. (2001) · doi:10.1093/acprof:oso/9780198524885.001.0001 · Cited by 553
Convolutional Neural Network Committees for Handwritten Character Classification
Ciresan, D. C., Meier, U., Gambardella, L. M., & Schmidhuber, J. (2011) · 2011 International Conference on Document Analysis and Recognition · doi:10.1109/icdar.2011.229 · Cited by 536
A Comparison of Primate Prefrontal and Inferior Temporal Cortices during Visual Categorization
Freedman, D. J., Riesenhuber, M., Poggio, T., & Miller, E. K. (2003) · The Journal of Neuroscience · doi:10.1523/jneurosci.23-12-05235.2003 · Cited by 537
“…figure shows a model of the recognition architecture in cortex ( Riesenhuber and Poggio , 1999 ) that combines and extends several recent models ( Fukushima , 1980 ; Perrett and Oram , 1993 ; Wallis and Rolls , 1997 ) and effectively summarizes many experimental find - ings ( for review , see…”
BackgroundDeep Learning over Multi-field Categorical Data
Zhang, W., Du, T., & Wang, J. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-30671-1_4 · Cited by 445
“The idea using FM in the bottom layer is ignited by Convolutional Neural Networks (CNNs) [11], which exploit spatially local correlation by enforcing a local connectivity pattern between neurons of adjacent layers.”
MethodMachine learning and deep learning—A review for ecologists
Pichler, M., & Hartig, F. (2023) · Methods in Ecology and Evolution · doi:10.1111/2041-210x.14061 · Cited by 454
“While many algorithms and network architectures that are used today were already described in the ‘80s and ‘90s (e.g. Fukushima, 1980; Lecun et al., 1998 ), their practical application was prevented by the lack of computing power at the time.”
The Temporal Correlation Hypothesis of Visual Feature Integration
Gray, C. M. (1999) · Neuron · doi:10.1016/s0896-6273(00)80820-x · Cited by 541
INVARIANT FACE AND OBJECT RECOGNITION IN THE VISUAL SYSTEM
Wallis, G., & Rolls, E. T. (1997) · Progress in Neurobiology · doi:10.1016/s0301-0082(96)00054-8 · Cited by 538
Latent feature representation with stacked auto-encoder for AD/MCI diagnosis
Initiative, T. A. D. N., Suk, H.-I., Lee, S.-W., & Shen, D. (2015) · Brain Structure and Function · doi:10.1007/s00429-013-0687-3 · Cited by 526
“Inspired from the biological model of the human visual cortex (Fukushima 1980; Serre et al. 2005), recent studies in machine learning have shown that a deep or hierarchical architecture is useful to find highly non-linear and complex patterns in data (Bengio 2009).”
MethodAction snippets: How many frames does human action recognition require?
Schindler, K., & van Gool, L. (2008) · 2008 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2008.4587730 · Cited by 526
“This operation was originally introduced in [14, 23], and has been shown to yield better translation invariance and feature preservation than linear averaging [ 14 , 25].”
BackgroundMethodHernandez, L., Baladron, C., Aguiar, J. M., Carro, B., Sanchez-Esguevillas, A. J., Lloret, J., & Massana, J. (2014) · IEEE Communications Surveys & Tutorials · doi:10.1109/surv.2014.032014.00094 · Cited by 506
“Three years later, in Japan, Kunihiko Fakushima presented a network model that used visual recognition of patterns (Fakushima, 1980 [42]).”
BackgroundAdvanced Steel Microstructural Classification by Deep Learning Methods
Azimi, S. M., Britz, D., Engstler, M., Fritz, M., & Mücklich, F. (2018) · Scientific Reports · doi:10.1038/s41598-018-20037-5 · Cited by 472
“The basic concept of CNNs dates back to 1980[9, 28], but due to the increase in computational resources, available data and better learning algorithms, they have recently attained a new level of quality across a range of domains.”
BackgroundA scoping review of transfer learning research on medical image analysis using ImageNet
Morid, M. A., Borjali, A., & Del Fiol, G. (2021) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2020.104115 · Cited by 508
A global inventory of photovoltaic solar energy generating units
Kruitwagen, L., Story, K. T., Friedrich, J., Byers, L., Skillman, S., & Hepburn, C. (2021) · Nature · doi:10.1038/s41586-021-03957-7 · Cited by 481
von der Malsburg, C. (1999) · Neuron · doi:10.1016/s0896-6273(00)80825-9 · Cited by 501
Shape perception reduces activity in human primary visual cortex
Murray, S. O., Kersten, D., Olshausen, B. A., Schrater, P., & Woods, D. L. (2002) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.192579399 · Cited by 496
The Future of Memristors: Materials Engineering and Neural Networks
Sun, K., Chen, J., & Yan, X. (2021) · Advanced Functional Materials · doi:10.1002/adfm.202006773 · Cited by 482
Tamar, A., Wu, Y., Thomas, G., Levine, S., & Abbeel, P. (2017) · Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2017/700 · Cited by 484
Phoneme and word recognition in the auditory ventral stream
DeWitt, I., & Rauschecker, J. P. (2012) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1113427109 · Cited by 465
Hyperspectral Image Classification—Traditional to Deep Models: A Survey for Future Prospects
Ahmad, M., Shabbir, S., Roy, S. K., Hong, D., Wu, X., Yao, J., Khan, A. M., Mazzara, M., Distefano, S., & Chanussot, J. (2022) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2021.3133021 · Cited by 438
Learning from Noisy Large-Scale Datasets with Minimal Supervision
Veit, A., Alldrin, N., Chechik, G., Krasin, I., Gupta, A., & Belongie, S. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.696 · Cited by 461
“The model comprises a fully convolutional neural network [12, 18, 19] f with two classi-”
MethodAttention mechanism in neural networks: where it comes and where it goes
Soydaner, D. (2022) · Neural Computing and Applications · doi:10.1007/s00521-022-07366-3 · Cited by 414
“One of the early studies is the improved version of the Neocognitron [20] with selective attention [21].”
BackgroundMixed Pooling for Convolutional Neural Networks
Yu, D., Wang, H., Chen, P., & Wei, Z. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-11740-9_34 · Cited by 409
“Since its first introduction in the early 1980’s [1], the Convolutional Neural Network (CNN) has demonstrated excellent performances for a number of applications such as hand-written digit recognition [2], face recognition [3], etc.”
BackgroundHoussein, E. H., Hammad, A., & Ali, A. A. (2022) · Neural Computing and Applications · doi:10.1007/s00521-022-07292-4 · Cited by 426
“Perhaps the first artificial neural network was the neocognitron [172] to have the ‘‘deep’’ property and take neurophysiological insights into consideration.”
BackgroundPredicting Parameters in Deep Learning
Shakibi, B. (2014) · University of British Columbia · doi:10.14288/1.0165555 · Cited by 462
Deep Multitask Learning for Railway Track Inspection
Gibert, X., Patel, V. M., & Chellappa, R. (2017) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2016.2568758 · Cited by 441
“Related works on inspection of railway tracks using computer vision are discussed in section II.”
Multiclass Object Recognition with Sparse, Localized Features
Mutch, J., & Lowe, D. G. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 1 (CVPR'06) · doi:10.1109/cvpr.2006.200 · Cited by 453
“These are the latest of a group of models which can be said to implement parts of the standard model, including convolutional networks [16] and Neocognitrons [10].”
BackgroundUnsupervised Feature Learning and Deep Learning: A Review and New Perspectives
Bengio, Y., Courville, A. C., & Vincent, P. (2012) · Cited by 581
Artificial intelligence in radiation oncology
Huynh, E., Hosny, A., Guthier, C., Bitterman, D. S., Petit, S. F., Haas-Kogan, D. A., Kann, B., Aerts, H. J. W. L., & Mak, R. H. (2020) · Nature Reviews Clinical Oncology · doi:10.1038/s41571-020-0417-8 · Cited by 419
Advancements in Image Classification using Convolutional Neural Network
Sultana, F., Sufian, A., & Dutta, P. (2018) · 2018 Fourth International Conference on Research in Computational Intelligence and Communication Networks (ICRCICN) · doi:10.1109/icrcicn.2018.8718718 · Cited by 390
“Motivated by this work, in 1980, Kunihiko Fukushima introduced neocognitron [5] which is a multi-layered neural network capable of recognizing”
MethodMachine Learning for Chemical Reactions
Meuwly, M. (2021) · Chemical Reviews · doi:10.1021/acs.chemrev.1c00033 · Cited by 437
Takahashi, S., Sakaguchi, Y., Kouno, N., Takasawa, K., Ishizu, K., Akagi, Y., Aoyama, R., Teraya, N., Bolatkan, A., Shinkai, N., et al. (2024) · Journal of Medical Systems · doi:10.1007/s10916-024-02105-8 · Cited by 343
“The concept of convolution in neural networks dates back to the 1980s with the introduction of the neocognitron by Kunihiko Fukushima in 1980 [41].”
Bock, F. E., Aydin, R. C., Cyron, C. J., Huber, N., Kalidindi, S. R., & Klusemann, B. (2019) · Frontiers in Materials · doi:10.3389/fmats.2019.00110 · Cited by 403
“Convolutional neural networks (CNNs) are mostly formulated as a variant of FFNNs and were pioneered in 1980 (Fukushima, 1980), and then reformulated in their contemporary form in 1999 (LeCun et al., 1999).”
BackgroundDeformable part models are convolutional neural networks
Girshick, R., Iandola, F., Darrell, T., & Malik, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298641 · Cited by 411
“Convolutional neural networks (CNNs) [12, 23, 27] are another influential class of models for visual recognition.”
BackgroundAnalyzing the Performance of Multilayer Neural Networks for Object Recognition
Agrawal, P., Girshick, R., & Malik, J. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10584-0_22 · Cited by 427
“Over the last two years, a sequence of results on benchmark visual recognition tasks has demonstrated that convolutional neural networks (CNNs) [6,14,18] will likely replace engineered features, such as SIFT [15] and HOG [2], for a wide variety of problems.”
BackgroundNeural network models for pattern recognition and associative memory
Carpenter, G. A. (1989) · Neural Networks · doi:10.1016/0893-6080(89)90035-x · Cited by 418
Pourbabaee, B., Roshtkhari, M. J., & Khorasani, K. (2018) · IEEE Transactions on Systems, Man, and Cybernetics: Systems · doi:10.1109/tsmc.2017.2705582 · Cited by 411
“CNNs were initially developed in the 1980s by Fukushima [33].”
BackgroundPerceiver: General Perception with Iterative Attention
Jaegle, A., Gimeno, F., Brock, A., Zisserman, A., Vinyals, O., & Carreira, J. (2021) · arXiv · doi:10.48550/arxiv.2103.03206 · Cited by 414
“ConvNets (Fukushima, 1980; LeCun et al., 1998; Cires¸an et al., 2011; Krizhevsky et al., 2012) have been the dominant family of architectures for perceptual tasks for nearly a full decade, thanks to their good performance and scalability.”
BackgroundStacked Convolutional Denoising Auto-Encoders for Feature Representation
Du, B., Xiong, W., Wu, J., Zhang, L., Zhang, L., & Tao, D. (2017) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2016.2536638 · Cited by 416
“The other property is convolution [16] and pooling [17] structures.”
BackgroundConvolutional Neural Networks for Image-Based High-Throughput Plant Phenotyping: A Review
Jiang, Y., & Li, C. (2020) · Plant Phenomics · doi:10.34133/2020/4152816 · Cited by 380
“CNNs originated in the1980s [14] and showed their first success in the recognition of handwritten digits in the 1990s by using backpropagationPostharvest Pl an t p he no ty pi ng Morphology”
MethodDeep learning for medical image segmentation: State-of-the-art advancements and challenges
Rayed, M. E., Islam, S. M. S., Niha, S. I., Jim, J. R., Kabir, M. M., & Mridha, M. F. (2024) · Informatics in Medicine Unlocked · doi:10.1016/j.imu.2024.101504 · Cited by 332
Metadata
- Authors
- Fukushima, K.
- Affiliations
- Japan Broadcasting Corporation (Japan) (JP)
- Corresponding author
- Fukushima, Kunihiko
- ORCID
- Fukushima, Kunihiko
- Venue
- Biological Cybernetics
- Published
- 1980-04
- Volume
- 36
- Issue
- 4
- Pages
- 193-202
- ISSN
- 0340-1200 (ISSN-L)
- 1432-0770
- OpenAlex
- W613766092
- Semantic Scholar
- 69e68bfaadf2dccff800158749f5a50fe82d173b
- Cited by
- 5,759
- References
- 13
- Citations by year
- 2026: 219 · 2025: 454 · 2024: 494 · 2023: 455 · 2022: 452 · 2021: 544 · 2020: 545 · 2019: 433 · 2018: 301 · 2017: 319 · 2016: 287 · 2015: 176 · 2014: 140 · 2013: 78 · 2012: 71
- FWCI
- 4.02
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.springer.com/tdm (tdm)
- Locations
- Biological Cybernetics (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Life Sciences › Neuroscience › Cognitive Neuroscience › Visual perception and processing mechanisms
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural dynamics and brain function
- Keywords
- neocognitron, visual pattern recognition, unsupervised learning, self-organization, position invariance, S cells, C cells, simple cells, complex cells, hypercomplex cells, visual cortex model, synaptic plasticity, shape recognition
- MeSH terms
- Cognition, Computers, Form Perception, Learning, Models, Neurological, Nerve Net, Nervous System Physiological Phenomena, Pattern Recognition, Visual
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.