Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position
The DOI matches its Crossref record; 12 of the 13 references it cites were found in a registry or an index.
Abstract
A neural network model for a mechanism of visual pattern recognition is proposed in this paper. The network is self-organized by "learning without a teacher", and acquires an ability to recognize stimulus patterns based on the geometrical similarity (Gestalt) of their shapes without affected by their positions. This network is given a nickname "neocognitron". After completion of self-organization, the network has a structure similar to the hierarchy model of the visual nervous system proposed by Hubel and Wiesel. The network consists of an input layer (photoreceptor array) followed by a cascade connection of a number of modular structures, each of which is composed of two layers of cells connected in a cascade. The first layer of each module consists of "S-cells", which show characteristics similar to simple cells or lower order hypercomplex cells, and the second layer consists of "C-cells" similar to complex cells or higher order hypercomplex cells. The afferent synapses to each S-cell have plasticity and are modifiable. The network has an ability of unsupervised learning: We do not need any "teacher" during the process of self-organization, and it is only needed to present a set of stimulus patterns repeatedly to the input layer of the network. The network has been simulated on a digital computer. After repetitive presentation of a set of stimulus patterns, each stimulus pattern has become to elicit an output only from one of the C-cells of the last layer, and conversely, t…
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Beyond the single-unit design discussed here, artificial neural networks were inspired by another seemingly universal design principle associated with cortical architecture [255]: the presence of multiple processing layers (such as those found in the visual cortex).”
From Fundamental constraints to the logic of living systems (2024) · cites this paper as [255]
In the citing paper
“Convolutional neural networks (CNNs) constitute one such class of models.9,15,17,19,21,26,32”
From ImageNet classification with deep convolutional neural networks (2017) · cites this paper as [9]
In the citing paper
“In the early 1980s, three new classes of NNs emerged, namely Convolutional Neural Network (CNN) [157], Self-Organizing Map (SOM) [249], and Hopfield network [195].”
From A comprehensive survey on machine learning for networking: Evolution, applications and research opportunities (2018) · cites this paper as [157]
Citations
Cites
Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex
Hubel, D. H., & Wiesel, T. N. (1962) · The Journal of Physiology · doi:10.1113/jphysiol.1962.sp006837 · Cited by 13,680
Ferrier lecture. Functional architecture of macaque monkey visual cortex
Hubel, D. H., & Wiesel, T. N. (1977) · Proceedings of the Royal Society of London. Series B. Biological Sciences · doi:10.1098/rspb.1977.0085 · Cited by 2,608
RECEPTIVE FIELDS AND FUNCTIONAL ARCHITECTURE IN TWO NONSTRIATE VISUAL AREAS (18 AND 19) OF THE CAT
Hubel, D. H., & Wiesel, T. N. (1965) · Journal of Neurophysiology · doi:10.1152/jn.1965.28.2.229 · Cited by 2,603
Mullin, A. A., & Rosenblatt, F. (1963) · The American Mathematical Monthly · doi:10.2307/2312103 · Cited by 2,209
Visual properties of neurons in inferotemporal cortex of the Macaque.
Gross, C. G., Rocha-Miranda, C. E., & Bender, D. B. (1972) · Journal of Neurophysiology · doi:10.1152/jn.1972.35.1.96 · Cited by 1,483 · cited 1×
“), and their responses are scarcely affected by the position or the size of the stimuli (Gross et al., 1972; Sato et al., 1978).”
BackgroundCognitron: A self-organizing multilayered neural network
Fukushima, K. (1975) · Biological Cybernetics · doi:10.1007/bf00342633 · Cited by 757
Self-citationde la Tour, E. B., Bolle, A., & Kellenberger, E. (2008) · Pathologia et Microbiologia · doi:10.1159/000161635 · Cited by 147
A Proposed Model for Visual Information Processing in the Human Brain
Snider, R. S. (1967) · Neurology · doi:10.1212/wnl.17.6.625 · Cited by 52
A Proposed Model for Visual Information Processing in the Human Brain
Kabrisky, M. (1966) · Illinois Digital Environment for Access to Learning and Scholarship (University of Illinois at Urbana-Champaign) · Cited by 44
Feature Extraction and Recognition of Handwritten Characters by Homogeneous Layers
Giebel, H. (1971) · Springer eBooks · doi:10.1007/978-3-642-65175-5_15 · Cited by 13
Research works printed but not linked: 4
- Fukushima, K.: Improvement in pattern-selectivity of a cognitron (in Japanese). Pap. Tech. Group MBE78-27, IECE Japan (1978)
- Fukushima, K.: Self-organization of a neural network which gives position-invariant response. In: Proceedings of the Sixth International Joint Conference on Artificial Intelligence. Tokyo, August 20?23, 1979, pp. 291?293 (1979b)
- Fukushima, K.: Improvement in pattern-selectivity of a cognitron (in Japanese). Trans. IECE Japan (A), J 62-A, 650?657 (1979c)
- Meyer, R.L., Sperry, R.W.: Explanatory models for neuroplasticity in retinotectral connections. In: Plasticity and function in the central nervous system. Stein, D.G., Rosen, J.J., Butters, N. (eds.), pp. 45?63. New York, San Francisco, London: Academic Press 1974
Cited by
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.81 · Cited by 32,684
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357
Deep learning in neural networks: an overview
Schmidhuber, J. (2015) · Neural Networks · doi:10.1016/j.neunet.2014.09.003 · Cited by 18,570
“For downsampling purposes he used Spatial Averaging (Fukushima, 1980, 2011) instead of Max-Pooling (MP, Section 5.11), currently a particularly convenient and popularWTA mechanism.”
BackgroundMethodInfluentialA survey on deep learning in medical image analysis
Litjens, G., Kooi, T., Bejnordi, B. E., Setio, A. A. A., Ciompi, F., Ghafoorian, M., van der Laak, J. A. W. M., van Ginneken, B., & Sánchez, C. I. (2017) · Medical Image Analysis · doi:10.1016/j.media.2017.07.005 · Cited by 15,332
“Work on CNNs has been done since the late sev-enties (Fukushima, 1980) and they were already applied to medical image analysis in 1995 by Lo et al. (1995).”
MethodRepresentation Learning: A Review and New Perspectives
Bengio, Y., Courville, A., & Vincent, P. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.50 · Cited by 13,001
“…(alternating selective feature extraction and invariance-creating pooling) has been the basis of convolutional networks, the Neocognitron (Fukushima, 1980) and HMAX (Riesenhuber and Poggio, 1999) models, and argued to be the architecture used by mammalian brains for object recognition…”
BackgroundBackpropagation Applied to Handwritten Zip Code Recognition
LeCun, Y., Boser, B., Denker, J. S., Henderson, D., Howard, R. E., Hubbard, W., & Jackel, L. D. (1989) · Neural Computation · doi:10.1162/neco.1989.1.4.541 · Cited by 11,891
Mastering the game of Go without human knowledge
Silver, D., Schrittwieser, J., Simonyan, K., Antonoglou, I., Huang, A., Guez, A., Hubert, T., Baker, L., Lai, M., Bolton, A., et al. (2017) · Nature · doi:10.1038/nature24270 · Cited by 9,084
“The neural network consists of many residual blocks 4 of convolutional layers 16,17 with batch normalization 18 and rectifier nonlinearities 19 (see Methods).”
BackgroundLearning Transferable Architectures for Scalable Image Recognition
Zoph, B., Vasudevan, V., Shlens, J., & Le, Q. V. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00907 · Cited by 6,249
Recent advances in convolutional neural networks
Gu, J., Wang, Z., Kuen, J., Ma, L., Shahroudy, A., Shuai, B., Liu, T., Wang, X., Wang, G., Cai, J., et al. (2018) · Pattern Recognition · doi:10.1016/j.patcog.2017.10.013 · Cited by 6,576
3D Convolutional Neural Networks for Human Action Recognition
Ji, S., Xu, W., Yang, M., & Yu, K. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.59 · Cited by 6,235
A Review of Recurrent Neural Networks: LSTM Cells and Network Architectures
Yu, Y., Si, X., Hu, C., & Zhang, J. (2019) · Neural Computation · doi:10.1162/neco_a_01199 · Cited by 5,591
Context Encoders: Feature Learning by Inpainting
Pathak, D., Krahenbuhl, P., Donahue, J., Darrell, T., & Efros, A. A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.278 · Cited by 5,691
“Recently, Convolutional Neural Networks (CNNs) [13, 27] have greatly advanced the performance in these tasks [15, 26,28].”
BackgroundA Survey of Convolutional Neural Networks: Analysis, Applications, and Prospects
Li, Z., Liu, F., Yang, W., Peng, S., & Zhou, J. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3084827 · Cited by 5,273
“As a result, pooling [22] (a.”
Learning Deep Architectures for AI
Bengio, Y. (2009) · now publishers, Inc. eBooks · doi:10.1561/9781601982957 · Cited by 4,941
Convolutional neural networks: an overview and application in radiology
Yamashita, R., Nishio, M., Do, R. K. G., & Togashi, K. (2018) · Insights into Imaging · doi:10.1007/s13244-018-0639-9 · Cited by 4,682
“1) CNN is a type of deep learning model for processing data that has a grid pattern, such as images, which is inspired by the organization of animal visual cortex [13, 14] and designed to automatically and adaptively learn spatial hierarchies of features, from low-to high-level patterns.”
BackgroundInception-v4, Inception-ResNet and the Impact of Residual Connections on Learning
Szegedy, C., Ioffe, S., Vanhoucke, V., & Alemi, A. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.11231 · Cited by 4,636
“Convolutional neural networks go back to the 1980s (Fukushima 1980) and (LeCun et al.”
Multi-column deep neural networks for image classification
Ciresan, D., Meier, U., & Schmidhuber, J. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6248110 · Cited by 3,720
Bootstrap Your Own Latent: A New Approach to Self-Supervised Learning
Grill, J.-B., Strub, F., Altché, F., Tallec, C., Richemond, P. H., Buchatskaya, E., Doersch, C., Pires, B. A., Guo, Z. D., Azar, M. G., et al. (2020) · arXiv · doi:10.48550/arxiv.2006.07733 · Cited by 3,846
“Learning good image representations is a key challenge in computer vision [1, 2, 3] as it allows for efficient training on downstream tasks [4, 5, 6, 7].”
BackgroundImage segmentation using deep learning: A survey
Minaee, S., Boykov, Y. Y., Porikli, F., Plaza, A. J., Kehtarnavaz, N., & Terzopoulos, D. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3059968 · Cited by 3,796
Deep Convolutional Neural Networks for Image Classification: A Comprehensive Review
Rawat, W., & Wang, Z. (2017) · Neural Computation · doi:10.1162/neco_a_00990 · Cited by 3,503
“irrespective of a shift in the position or considerable distortion in the shape of the input pattern (Fukushima, 1980; Fukushima & Miyake, 1982).”
BackgroundInfluentialDeep learning for computer vision: A brief review
Voulodimos, A., Doulamis, N., Doulamis, A., & Protopapadakis, E. (2018) · Computational Intelligence and Neuroscience · doi:10.1155/2018/7068349 · Cited by 3,403
“The first computational models based on these local connectivities between neurons and on hierarchically organized transformations of the image are found in Neocognitron [19], which describes that when neurons with the same parameters are applied on patches of the previous layer at different locations, a form of trans…”
BackgroundHierarchical models of object recognition in cortex
Riesenhuber, M., & Poggio, T. (1999) · Nature Neuroscience · doi:10.1038/14819 · Cited by 3,309
Dueling Network Architectures for Deep Reinforcement Learning
Wang, Z., Schaul, T., Hessel, M., van Hasselt, H., Lanctot, M., & de Freitas, N. (2015) · arXiv · doi:10.48550/arxiv.1511.06581 · Cited by 3,300
“Rectifier non-linearities (Fukushima, 1980) are inserted between all adjacent layers.”
BackgroundConvolutional Neural Networks for Medical Image Analysis: Full Training or Fine Tuning?
Tajbakhsh, N., Shin, J. Y., Gurudu, S. R., Hurst, R. T., Kendall, C. B., Gotway, M. B., & Liang, J. (2016) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2016.2535302 · Cited by 3,277
“C ONVOLUTIONAL neural networks (CNNs) have been used in the field of computer vision for decades [1]–[3].”
MethodA survey of deep neural network architectures and their applications
Liu, W., Wang, Z., Liu, X., Zeng, N., Liu, Y., & Alsaadi, F. E. (2017) · Neurocomputing · doi:10.1016/j.neucom.2016.12.038 · Cited by 3,239
“model Neocognitron was introduced in [46] with hierarchica lly organized image transformations.”
MethodLearning and Transferring Mid-level Image Representations Using Convolutional Neural Networks
Oquab, M., Bottou, L., Laptev, I., & Sivic, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.222 · Cited by 3,150
“We also show promising results for object and action localization.”
Survey on deep learning with class imbalance
Johnson, J. M., & Khoshgoftaar, T. M. (2019) · Journal of Big Data · doi:10.1186/s40537-019-0192-5 · Cited by 3,032
Learning internal representations by error propagation
RUMELHART, D. E., HINTON, G. E., & WILLIAMS, R. J. (1988) · Readings in Cognitive Science · doi:10.1016/b978-1-4832-1446-7.50035-2 · Cited by 3,002
Region-based convolutional networks for accurate object detection and segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2016) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2437384 · Cited by 2,912
“We trace the roots of our approach to Fukushima's " neocognitron " [4], a hierarchical and shift-invariant model for pattern recognition.”
MethodDeep learning-based classification of hyperspectral data
Chen, Y., Lin, Z., Zhao, X., Wang, G., & Gu, Y. (2014) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2014.2329330 · Cited by 2,679
“The layer-wise training models have a bunch of alternatives such as restricted Boltzmann machines (RBMs) [42], pooling units [43], convolutional neural networks (CNNs) [44], AEs, and denoising AEs (DAE) [40].”
MethodEnd-to-End Training of Deep Visuomotor Policies
Levine, S., Finn, C., Darrell, T. J., & Abbeel, P. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1504.00702 · Cited by 1,344
“CNNs have a long history in computer vision and deep learning (Fukushima, 1980; LeCun et al., 1989; Schmidhuber, 2015), and have recently gained prominence due to excellent results on a number of vision benchmarks (Ciresan et al.”
BackgroundNeuronal Synchrony: A Versatile Code for the Definition of Relations?
Singer, W. (1999) · Neuron · doi:10.1016/s0896-6273(00)80821-1 · Cited by 2,581
Building machines that learn and think like people
Lake, B. M., Ullman, T. D., Tenenbaum, J. B., & Gershman, S. J. (2017) · Behavioral and Brain Sciences · doi:10.1017/s0140525x16001837 · Cited by 2,550
Building Machines That Learn and Think Like People
Lake, B. M., Ullman, T. D., Tenenbaum, J. B., & Gershman, S. J. (2016) · arXiv · doi:10.48550/arxiv.1604.00289 · Cited by 2,550
A massively parallel architecture for a self-organizing neural pattern recognition machine
Carpenter, G. A., & Grossberg, S. (1987) · Computer Vision, Graphics, and Image Processing · doi:10.1016/s0734-189x(87)80014-2 · Cited by 2,525
Single-Image Crowd Counting via Multi-Column Convolutional Neural Network
Zhang, Y., Zhou, D., Chen, S., Gao, S., & Ma, Y. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.70 · Cited by 2,241
“To overcome above challenges, in this work, we propose a novel framework based on convolutional neural network (CNN) [9, 16] for crowd counting in an arbitrary still image.”
MethodConvolutional Neural Networks for Speech Recognition
Abdel-Hamid, O., Mohamed, A.-R., Jiang, H., Deng, L., Penn, G., & Yu, D. (2014) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2014.2339736 · Cited by 2,262
“See http://www.ieee.org/publications_standards/publications/rights/index.html for more information. modeling because training ANNs with so many hidden units on so many hours of speech data has only recently become feasible.”
Background30 years of adaptive neural networks: perceptron, Madaline, and backpropagation
Widrow, B., & Lehr, M. A. (1990) · Proceedings of the IEEE · doi:10.1109/5.58323 · Cited by 2,242
Stacked Convolutional Auto-Encoders for Hierarchical Feature Extraction
Masci, J., Meier, U., Cireşan, D., & Schmidhuber, J. (2011) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-21735-7_7 · Cited by 1,955
“In the context of neural network architectures, unsupervised layers can be stacked on top of each other to build deep hierarchies [7].”
BackgroundComputing with Neural Circuits: A Model
Hopfield, J. J., & Tank, D. W. (1986) · Science · doi:10.1126/science.3755256 · Cited by 2,116
Deep Learning--based Text Classification: A Comprehensive Review
Minaee, S., Kalchbrenner, N., Cambria, E., Nikzad, N., Chenaghlu, M., & Gao, J. (2022) · ACM Computing Surveys · doi:10.1145/3439726 · Cited by 1,538
Performance-optimized hierarchical models predict neural responses in higher visual cortex
Yamins, D. L. K., Hong, H., Cadieu, C. F., Solomon, E. A., Seibert, D., & DiCarlo, J. J. (2014) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1403112111 · Cited by 1,924
“Proc Natl Acad Sci USA 104(15):6424–6429.”
Bergstra, J., Yamins, D. L. K., & Cox, D. (2013) · Digital Access to Scholarship at Harvard (DASH) (Harvard University) · Cited by 1,852
A Survey of the Usages of Deep Learning for Natural Language Processing
Otter, D. W., Medina, J. R., & Kalita, J. K. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.2979670 · Cited by 1,836
“networks (CNNs) [14], [15], built upon Fukashima’s neocognitron [16], [17], derive the name from the convolution operation in mathematics and signal processing.”
BackgroundHow does the brain solve visual object recognition?
DiCarlo, J. J., Zoccolan, D., & Rust, N. C. (2012) · Neuron · doi:10.1016/j.neuron.2012.01.010 · Cited by 1,790
“For example, while the algorithms of (Fukushima, 1980; Riesenhuber and Poggio, 1999b; Serre et al., 2007a) represent a great start, we also know that they are insufficient in that they perform only slightly better than baseline V1-like benchmark algorithms (Pinto et al.”
InfluentialAcharya, U. R., Oh, S. L., Hagiwara, Y., Tan, J. H., & Adeli, H. (2018) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2017.09.017 · Cited by 1,647
“However, the ANN model is susceptible to shift and translation distortion which may result in poor classification accuracy [21].”
BackgroundChallenges in Representation Learning: A Report on Three Machine Learning Contests
Goodfellow, I. J., Erhan, D., Carrier, P. L., Courville, A., Mirza, M., Hamner, B., Cukierski, W., Tang, Y., Thaler, D., Lee, D.-H., et al. (2013) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-42051-1_16 · Cited by 1,719
Barlow, H. B. (1989) · Neural Computation · doi:10.1162/neco.1989.1.3.295 · Cited by 1,650
Robust Object Recognition with Cortex-Like Mechanisms
Serre, T., Wolf, L., Bileschi, S., Riesenhuber, M., & Poggio, T. (2007) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2007.56 · Cited by 1,667
“Multilayered convolutional networks were shown to perform extremely well in the domain of digit recognition [4], [5] and, more recently, generic object recognition [10] and face identification [22].”
BackgroundMethodUsing goal-driven deep learning models to understand sensory cortex
Yamins, D. L. K., & DiCarlo, J. J. (2016) · Nature Neuroscience · doi:10.1038/nn.4244 · Cited by 1,621
“11–15).”
Algorithm Unrolling: Interpretable, Efficient Deep Learning for Signal and Image Processing
Monga, V., Li, Y., & Eldar, Y. C. (2021) · IEEE Signal Processing Magazine · doi:10.1109/msp.2020.3016905 · Cited by 1,414
“[19] designed a neural network by mimicking the visual nervous system [20].”
BackgroundDeep learning for universal linear embeddings of nonlinear dynamics
Lusch, B., Kutz, J. N., & Brunton, S. L. (2018) · Nature Communications · doi:10.1038/s41467-018-07210-0 · Cited by 1,424
Neuroscience-Inspired Artificial Intelligence
Hassabis, D., Kumaran, D., Summerfield, C., & Botvinick, M. (2017) · Neuron · doi:10.1016/j.neuron.2017.06.011 · Cited by 1,557
Mitosis Detection in Breast Cancer Histology Images with Deep Neural Networks
Cireşan, D. C., Giusti, A., Gambardella, L. M., & Schmidhuber, J. (2013) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-40763-5_51 · Cited by 1,536
A Survey on Deep Learning: Algorithms, Techniques, and Applications
Pouyanfar, S., Sadiq, S., Yan, Y., Tian, H., Tao, Y., Reyes, M. P., Shyu, M.-L., Chen, S.-C., & Iyengar, S. S. (2019) · ACM Computing Surveys · doi:10.1145/3234150 · Cited by 1,403
“In 1980, “neocogitron,” which inspired the convolutional neural network, was introduced [40], while Recurrent Neural Networks (RNNs) were proposed in 1986 [73].”
BackgroundArtificial intelligence to deep learning: machine intelligence approach for drug discovery
Gupta, R., Srivastava, D., Sahu, M., Tiwari, S., Ambasta, R. K., & Kumar, P. (2021) · Molecular Diversity · doi:10.1007/s11030-021-10217-3 · Cited by 1,473
“From the past two decades, the development of efficient and advanced systems for the targeted delivery of therapeutic agents with maximum efficiency and minimum risks has imposed a great challenge among chemical and biological scientists [1].”
BackgroundInfluentialEnhancing the Locality and Breaking the Memory Bottleneck of Transformer on Time Series Forecasting
Li, S., Jin, X., Xuan, Y., Zhou, X., Chen, W., Wang, Y.-X., & Yan, X. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.00235 · Cited by 1,053
Target Classification Using the Deep Convolutional Networks for SAR Images
Chen, S., Wang, H., Xu, F., & Jin, Y.-Q. (2016) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2016.2551720 · Cited by 1,347
“The lack of fully connected layers resembles the architecture of Fukushima’s neocognitron [34], which is considered as an origin of the ConvNets.”
BackgroundVisual saliency based on multiscale deep features
Li, G., & Yu, Y. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299184 · Cited by 1,163
“…neural networks (CNN) are particularly well suited for this task because convolutional layers in a CNN resemble simple and complex cells in the human visual system [14] while fully connected layers in a CNN resemble higher-level inference and decision making in the human cognitive system.”
Background(2012) · neural information processing systems · Cited by 1,291
Data-Driven Science and Engineering: Machine Learning, Dynamical Systems, and Control
Brunton, S. L., & Kutz, J. N. (2019) · Cambridge University Press eBooks · doi:10.1017/9781108380690 · Cited by 1,154
Spectral-spatial classification of hyperspectral data based on deep belief network
Chen, Y., Zhao, X., & Jia, X. (2015) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2015.2388577 · Cited by 1,259
“Since the responses of all the spectral channels are used as input data, we have to regularize the model for sparsity [41].”
Ahmed, R., Sreeram, V., Mishra, Y., & Arif, M. D. (2020) · Renewable and Sustainable Energy Reviews · doi:10.1016/j.rser.2020.109792 · Cited by 1,259
“The computational model utilized is called Neocognitron [223] which relies on linear or nonlinear filters to extract the features from an image.”
MethodDeep Machine Learning - A New Frontier in Artificial Intelligence Research [Research Frontier]
Arel, I., Rose, D. C., & Karnowski, T. P. (2010) · IEEE Computational Intelligence Magazine · doi:10.1109/mci.2010.938364 · Cited by 1,121
“Similar organizations are utilized by CNNs as well as other deep-layered models (such as the Neocognitron [40] [41] [43] and HMAX [32] [45]), yet”
MethodDollar, P., Tu, Z., Perona, P., & Belongie, S. (2009) · Procedings of the British Machine Vision Conference 2009 · doi:10.5244/c.23.91 · Cited by 1,237
“Integral Histogram: Porikli [20] presented an efficient method for computing histograms using integral images.”
BackgroundMethodInfluential(2011) · doi:10.5591/978-1-57735-516-8/ijcai11-210 · Cited by 1,229
Bao, W., Yue, J., & Rao, Y. (2017) · PLOS ONE · doi:10.1371/journal.pone.0180944 · Cited by 1,124
“Restricted Boltzmann machines (RBMs) [60], convolutional neural networks (CNNs) [61], and autoencoders[14] are the frequently used layer-wise training models.”
BackgroundHu, F., Xia, G.-S., Hu, J., & Zhang, L. (2015) · Remote Sensing · doi:10.3390/rs71114680 · Cited by 1,195
“Although we have recently witnessed the overwhelming popularity of CNNs, the original concept of CNN dates back to Fukushima’s biologically inspired “neocognitron” [41], a hierarchical network with invariance to image translations.”
BackgroundPansharpening by Convolutional Neural Networks
Masi, G., Cozzolino, D., Verdoliva, L., & Scarpa, G. (2016) · Remote Sensing · doi:10.3390/rs8070594 · Cited by 1,152
“Convolutional neural networks (CNN) [37] overcome to a large extent this problem by renouncing full connectivity.”
BackgroundSimultaneous Detection and Segmentation
Hariharan, B., Arbeláez, P., Girshick, R., & Malik, J. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10584-0_20 · Cited by 1,085
“CNNs have their roots in the Neocogni-tron proposed by Fukushima [15].”
BackgroundOverview of deep learning in medical imaging
Suzuki, K. (2017) · Radiological Physics and Technology · doi:10.1007/s12194-017-0406-5 · Cited by 1,087
“A CNN can be considered as a simplified version of the Neocognitron model that was proposed to simulate the human visual system in 1980 by Fukushima [50].”
BackgroundInfluentialNeocognitron: A hierarchical neural network capable of visual pattern recognition
Fukushima, K. (1988) · Neural Networks · doi:10.1016/0893-6080(88)90014-7 · Cited by 1,097
Self-citationConvolutional neural network: a review of models, methodologies and applications to object detection
Dhillon, A., & Verma, G. K. (2020) · Progress in Artificial Intelligence · doi:10.1007/s13748-019-00203-0 · Cited by 1,119
Abràmoff, M. D., Lou, Y., Erginay, A., Clarida, W., Amelon, R., Folk, J. C., & Niemeijer, M. (2016) · Investigative Opthalmology & Visual Science · doi:10.1167/iovs.16-19964 · Cited by 1,101
Kriegeskorte, N. (2015) · Annual Review of Vision Science · doi:10.1146/annurev-vision-082114-035447 · Cited by 1,093
Deep Learning on Image Denoising: An overview
Tian, C., Fei, L., Zheng, W., Xu, Y., Zuo, W., & Lin, C.-W. (2020) · Neural Networks · doi:10.1016/j.neunet.2020.07.025 · Cited by 1,091
Fast-classifying, high-accuracy spiking deep networks through weight and threshold balancing
Diehl, P. U., Neil, D., Binas, J., Cook, M., Liu, S.-C., & Pfeiffer, M. (2015) · 2015 International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2015.7280696 · Cited by 991
The Correlation Theory of Brain Function
von der Malsburg, C. (1994) · Physics of Neural Networks · doi:10.1007/978-1-4612-4320-5_2 · Cited by 1,089
Multi-column deep neural network for traffic sign classification
Cireşan, D., Meier, U., Masci, J., & Schmidhuber, J. (2012) · Neural Networks · doi:10.1016/j.neunet.2012.02.023 · Cited by 1,057
“One of the first hierarchical neural systems was the Neocognitron by Fukushima (1980), which inspired many of the more recent variants.”
BackgroundClassification of breast cancer histology images using Convolutional Neural Networks
Araújo, T., Aresta, G., Castro, E., Rouco, J., Aguiar, P., Eloy, C., Polónia, A., & Campilho, A. (2017) · PLOS ONE · doi:10.1371/journal.pone.0177544 · Cited by 1,013
Deep Convolutional Neural Network Based Regression Approach for Estimation of Remaining Useful Life
Sateesh Babu, G., Zhao, P., & Li, X.-L. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-32025-0_14 · Cited by 962
“When these operators with the same parameters are applied on local signals (or their mapping) at different time segments, a form of translation invariance is obtained [2,7,8].”
BackgroundConvolutional Neural Networks for Diabetic Retinopathy
Pratt, H., Coenen, F., Broadbent, D. M., Harding, S. P., & Zheng, Y. (2016) · Procedia Computer Science · doi:10.1016/j.procs.2016.07.014 · Cited by 988
“Network architectures designed to work with image data were routinely built already in 1970s 10 with useful applications and surpassed other approaches to challenging tasks like handwritten character recognition 11 .”
MethodBoutaba, R., Salahuddin, M. A., Limam, N., Ayoubi, S., Shahriar, N., Estrada-Solano, F., & Caicedo, O. M. (2018) · Journal of Internet Services and Applications · doi:10.1186/s13174-018-0087-2 · Cited by 1,036
“In the early 1980s, three new classes of NNs emerged, namely Convolutional Neural Network (CNN) [157], Self-OrganizingMap (SOM) [249], and Hopfield network [195].”
BackgroundInfluentialRecurrent convolutional neural network for object recognition
Liang, M., & Hu, X. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298958 · Cited by 976
“Essentially CNN differs from the Neocognitron by incorporating the backpropagation (BP) algorithm for learning the receptive fields of simple units.”
BackgroundMethodInfluentialA deep learning framework for neuroscience
Richards, B. A., Lillicrap, T. P., Beaudoin, P., Bengio, Y., Bogacz, R., Christensen, A., Clopath, C., Costa, R. P., de Berker, A., Ganguli, S., et al. (2019) · Nature Neuroscience · doi:10.1038/s41593-019-0520-2 · Cited by 1,027
Environmental sound classification with convolutional neural networks
Piczak, K. J. (2015) · 2015 IEEE 25th International Workshop on Machine Learning for Signal Processing (MLSP) · doi:10.1109/mlsp.2015.7324337 · Cited by 931
“Index Terms— environmental sound, convolutional neural networks, classification 1.”
BackgroundDeep Learning Algorithms for Bearing Fault Diagnostics—A Comprehensive Review
Zhang, S., Zhang, S., Wang, B., & Habetler, T. G. (2020) · IEEE Access · doi:10.1109/access.2020.2972859 · Cited by 913
“Inspired by animal visual cortices [103], the convolution operation is first introduced to detect image patterns in a hierarchical way from simple features such as edge and corner to complex features.”
MethodUntangling invariant object recognition
DiCarlo, J. J., & Cox, D. D. (2007) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2007.06.010 · Cited by 959
Plant Disease Detection and Classification by Deep Learning
Saleem, M. H., Potgieter, J., & Arif, K. M. (2019) · Plants · doi:10.3390/plants8110468 · Cited by 886
“During the first phase, several developments like backpropagation [2,3], chain rule [4], Neocognitron [5], hand written text recognition (LeNET architecture) [6], and resolving the training problem [7,8] were observed (as shown in Figure 1).”
Neocognitron: A Self-Organizing Neural Network Model for a Mechanism of Visual Pattern Recognition
Fukushima, K., & Miyake, S. (1982) · Lecture Notes in Biomathematics · doi:10.1007/978-3-642-46466-9_18 · Cited by 991
Self-citation(2015) · Cited by 989
Stand-Alone Self-Attention in Vision Models
Ramachandran, P., Parmar, N., Vaswani, A., Bello, I., Levskaya, A., & Shlens, J. (2019) · arXiv · doi:10.48550/arxiv.1906.05909 · Cited by 977
Neural NILM: Deep Neural Networks Applied to Energy Disaggregation
Kelly, J., & Knottenbelt, W. (2015) · Proceedings of the 2nd ACM International Conference on Embedded Systems for Energy-Efficient Built Environments · doi:10.1145/2821650.2821672 · Cited by 891
“Convolutional neural nets (CNNs) [20, 21, 22] build a small number of filters, each with a small receptive field, and these filters are duplicated (with shared weights) across the entire input.”
BackgroundLarge scale deep learning for computer aided detection of mammographic lesions
Kooi, T., Litjens, G., van Ginneken, B., Gubern-Mérida, A., Sánchez, C. I., Mann, R., den Heeten, A., & Karssemeijer, N. (2017) · Medical Image Analysis · doi:10.1016/j.media.2016.07.007 · Cited by 965
Guclu, U., & van Gerven, M. A. J. (2015) · Journal of Neuroscience · doi:10.1523/jneurosci.5023-14.2015 · Cited by 961
Neural mechanisms for the recognition of biological movements
Giese, M. A., & Poggio, T. (2003) · Nature Reviews Neuroscience · doi:10.1038/nrn1057 · Cited by 955
Neural Networks and Deep Learning
Theodoridis, S. (2015) · Machine Learning · doi:10.1016/b978-0-12-801522-3.00018-5 · Cited by 900
Spiking Deep Convolutional Neural Networks for Energy-Efficient Object Recognition
Cao, Y., Chen, Y., & Khosla, D. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-014-0788-3 · Cited by 917
“Y. Cao · Y. Chen (B) · D. Khosla HRL Laboratories, LLC, 3011 Malibu Canyon Road, Malibu, CA 90265-4797, USA e-mail: ychen@hrl.com”
MethodSEMANTIC3D.NET: A NEW LARGE-SCALE POINT CLOUD CLASSIFICATION BENCHMARK
Hackel, T., Savinov, N., Ladicky, L., Wegner, J. D., Schindler, K., & Pollefeys, M. (2017) · ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences · doi:10.5194/isprs-annals-iv-1-w1-91-2017 · Cited by 812
Metadata
- Authors
- Fukushima, K.
- Affiliations
- Japan Broadcasting Corporation (Japan) (JP)
- Corresponding author
- Fukushima, Kunihiko
- ORCID
- Fukushima, Kunihiko
- Venue
- Biological Cybernetics
- Published
- 1980-04
- Volume
- 36
- Issue
- 4
- Pages
- 193-202
- ISSN
- 0340-1200 (ISSN-L)
- 1432-0770
- OpenAlex
- W613766092
- Semantic Scholar
- 69e68bfaadf2dccff800158749f5a50fe82d173b
- Cited by
- 5,759
- References
- 13
- Citations by year
- 2026: 219 · 2025: 454 · 2024: 494 · 2023: 455 · 2022: 452 · 2021: 544 · 2020: 545 · 2019: 433 · 2018: 301 · 2017: 319 · 2016: 287 · 2015: 176 · 2014: 140 · 2013: 78 · 2012: 71
- FWCI
- 4.02
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.springer.com/tdm (tdm)
- Locations
- Biological Cybernetics (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Life Sciences › Neuroscience › Cognitive Neuroscience › Visual perception and processing mechanisms
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural dynamics and brain function
- Keywords
- neocognitron, visual pattern recognition, unsupervised learning, self-organization, position invariance, S cells, C cells, simple cells, complex cells, hypercomplex cells, visual cortex model, synaptic plasticity, shape recognition
- MeSH terms
- Cognition, Computers, Form Perception, Learning, Models, Neurological, Nerve Net, Nervous System Physiological Phenomena, Pattern Recognition, Visual
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.