The DOI matches its Crossref record; 15 of the 15 references it cites were found in a registry or an index.
Self-organization in a perceptual network
Abstract
The emergence of a feature-analyzing function from the development rules of simple, multilayered networks is explored. It is shown that even a single developing cell of a layered network exhibits a remarkable set of optimization properties that are closely related to issues in statistics, theoretical physics, adaptive signal processing, the formation of knowledge representation in artificial intelligence, and information theory. The network studied is based on the visual system. These results are used to infer an information-theoretic principle that can be applied to the network as a whole, rather than a single cell. The organizing principle proposed is that the network connections develop in such a way as to maximize the amount of information that is preserved when signals are transformed at each processing stage, subject to certain constraints. The operation of this principle is illustrated for some simple cases.>
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A Mathematical Theory of Communication
Shannon, C. E. (1948) · Bell System Technical Journal · doi:10.1002/j.1538-7305.1948.tb01338.x · Cited by 88,486
A Mathematical Theory of Communication
Shannon, C. E. (1948) · Bell System Technical Journal · doi:10.1002/j.1538-7305.1948.tb00917.x · Cited by 48,949
Neural networks and physical systems with emergent collective computational abilities.
Hopfield, J. J. (1982) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.79.8.2554 · Cited by 19,270
Hebb, D. O. (2005) · doi:10.4324/9781410612403 · Cited by 9,825
Self-Organization and Associative Memory
Kohonen, T. (1989) · Springer Series in Information Sciences · doi:10.1007/978-3-642-88163-3 · Cited by 8,740
indicated, N. A. (1950) · Journal of Consulting Psychology · doi:10.1037/h0049612 · Cited by 6,923
Alexander, S. T. (1986) · Springer eBooks · doi:10.1007/978-1-4612-4978-8 · Cited by 2,646
Ferrier lecture. Functional architecture of macaque monkey visual cortex
Hubel, D. H., & Wiesel, T. N. (1977) · Proceedings of the Royal Society of London. Series B. Biological Sciences · doi:10.1098/rspb.1977.0085 · Cited by 2,608
Simplified neuron model as a principal component analyzer
Oja, E. (1982) · Journal of Mathematical Biology · doi:10.1007/bf00275687 · Cited by 2,309
Huber, P. J. (1985) · The Annals of Statistics · doi:10.1214/aos/1176349519 · Cited by 1,277
Cited by
Deep learning in neural networks: an overview
Schmidhuber, J. (2015) · Neural Networks · doi:10.1016/j.neunet.2014.09.003 · Cited by 18,570
Emergence of simple-cell receptive field properties by learning a sparse code for natural images
Olshausen, B. A., & Field, D. J. (1996) · Nature · doi:10.1038/381607a0 · Cited by 5,694
Probabilistic Independent Component Analysis for Functional Magnetic Resonance Imaging
Beckmann, C. F., & Smith, S. M. (2004) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2003.822821 · Cited by 2,715
The “independent components” of natural scenes are edge filters
Bell, A. J., & Sejnowski, T. J. (1997) · Vision Research · doi:10.1016/s0042-6989(97)00121-1 · Cited by 2,201
Functional Connectivity: The Principal-Component Analysis of Large (PET) Data Sets
Friston, K. J., Frith, C. D., Liddle, P. F., & Frackowiak, R. S. J. (1993) · Journal of Cerebral Blood Flow & Metabolism · doi:10.1038/jcbfm.1993.4 · Cited by 1,978
Optimal unsupervised learning in a single-layer linear feedforward neural network
Sanger, T. D. (1989) · Neural Networks · doi:10.1016/0893-6080(89)90044-0 · Cited by 1,509
Forward models: supervised learning with a distal teacher
Jordan, M. I., & Rumelhart, D. E. (1992) · Cognitive Science · doi:10.1207/s15516709cog1603_1 · Cited by 1,451
What Is the Goal of Sensory Coding?
Field, D. J. (1994) · Neural Computation · doi:10.1162/neco.1994.6.4.559 · Cited by 1,183
Rate, Timing, and Cooperativity Jointly Determine Cortical Synaptic Plasticity
Sjöström, P. J., Turrigiano, G. G., & Nelson, S. B. (2001) · Neuron · doi:10.1016/s0896-6273(01)00542-6 · Cited by 1,174
Learning Overcomplete Representations
Lewicki, M. S., & Sejnowski, T. J. (2000) · Neural Computation · doi:10.1162/089976600300015826 · Cited by 1,129
Alignment by maximization of mutual information
Viola, P., & Wells, W. M. (2002) · Proceedings of IEEE International Conference on Computer Vision · doi:10.1109/iccv.1995.466930 · Cited by 1,082
Graph Contrastive Learning with Adaptive Augmentation
Zhu, Y., Xu, Y., Yu, F., Liu, Q., Wu, S., & Wang, L. (2021) · Proceedings of the Web Conference 2021 · doi:10.1145/3442381.3449802 · Cited by 1,066
Wang, T., & Isola, P. (2020) · arXiv · doi:10.48550/arxiv.2005.10242 · Cited by 921
A Unifying Review of Deep and Shallow Anomaly Detection
Ruff, L., Kauffmann, J. R., Vandermeulen, R. A., Montavon, G., Samek, W., Kloft, M., Dietterich, T. G., & Muller, K.-R. (2021) · Proceedings of the IEEE · doi:10.1109/jproc.2021.3052449 · Cited by 914
Principal components, minor components, and linear neural networks
Oja, E. (1992) · Neural Networks · doi:10.1016/s0893-6080(05)80089-9 · Cited by 814
S3-Rec: Self-Supervised Learning for Sequential Recommendation with Mutual Information Maximization
Zhou, K., Wang, H., Zhao, W. X., Zhu, Y., Wang, S., Zhang, F., Wang, Z., & Wen, J.-R. (2020) · Proceedings of the 29th ACM International Conference on Information & Knowledge Management · doi:10.1145/3340531.3411954 · Cited by 789
What makes for good views for contrastive learning
Tian, Y., Sun, C., Poole, B., Krishnan, D., Schmid, C., & Isola, P. (2020) · arXiv · doi:10.48550/arxiv.2005.10243 · Cited by 773
Barlow Twins: Self-Supervised Learning via Redundancy Reduction
Žbontar, J., Jing, L., Misra, I., LeCun, Y., & Deny, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.03230 · Cited by 751
Dendritic Excitability and Synaptic Plasticity
Sjöström, P. J., Rancz, E. A., Roth, A., & Häusser, M. (2008) · Physiological Reviews · doi:10.1152/physrev.00016.2007 · Cited by 725
Could information theory provide an ecological theory of sensory processing?
Atick, J. J. (2011) · Network: Computation in Neural Systems · doi:10.3109/0954898x.2011.638888 · Cited by 679
Neural Networks and Related Methods for Classification
Ripley, B. D. (1994) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1994.tb01990.x · Cited by 627
Local feature analysis: a general statistical theory for object representation
Penev, P. S., & Atick, J. J. (1996) · Network Computation in Neural Systems · doi:10.1088/0954-898x/7/3/002 · Cited by 627
Natural image statistics and efficient coding
Olshausen, B. A., & Field, D. J. (1996) · Network: Computation in Neural Systems · doi:10.1088/0954-898x_7_2_014 · Cited by 561
Neuromodulation and cortical function: modeling the physiological basis of behavior
Hasselmo, M. E. (1995) · Behavioural Brain Research · doi:10.1016/0166-4328(94)00113-t · Cited by 556
Towards a Theory of Early Visual Processing
Atick, J. J., & Redlich, A. N. (1990) · Neural Computation · doi:10.1162/neco.1990.2.3.308 · Cited by 554
Computational Neuroscience of Vision
Rolls, E., & Deco, G. (2001) · doi:10.1093/acprof:oso/9780198524885.001.0001 · Cited by 553
Beyond Phrenology: What Can Neuroimaging Tell Us About Distributed Circuitry?
Friston, K. (2002) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.25.112701.142846 · Cited by 465
Self-supervised Heterogeneous Graph Neural Network with Co-contrastive Learning
Wang, X., Liu, N., Han, H., & Shi, C. (2021) · Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining · doi:10.1145/3447548.3467415 · Cited by 467
Hasselmo, M., & Schnell, E. (1994) · The Journal of Neuroscience · doi:10.1523/jneurosci.14-06-03898.1994 · Cited by 466
Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture
Assran, M., Duval, Q., Misra, I., Bojanowski, P., Vincent, P., Rabbat, M., LeCun, Y., & Ballas, N. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01499 · Cited by 456
Could information theory provide an ecological theory of sensory processing?
Atick, J. J. (1992) · Network: Computation in Neural Systems · doi:10.1088/0954-898x_3_2_009 · Cited by 446
Contrastive Multi-View Representation Learning on Graphs
Hassani, K., & Khasahmadi, A. H. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.05582 · Cited by 423
How independent are the messages carried by adjacent inferior temporal cortical neurons?
Gawne, T., & Richmond, B. (1993) · The Journal of Neuroscience · doi:10.1523/jneurosci.13-07-02758.1993 · Cited by 420
Vernon, D., Metta, G., & Sandini, G. (2007) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2006.890274 · Cited by 414
Information processing, dimensionality reduction and reinforcement learning in the basal ganglia
Bar-Gad, I., Morris, G., & Bergman, H. (2003) · Progress in Neurobiology · doi:10.1016/j.pneurobio.2003.12.001 · Cited by 399
The development of topography in the visual cortex: a review of models
Swindale, N. (1996) · Network: Computation in Neural Systems · doi:10.1088/0954-898x/7/2/002 · Cited by 398
A Bayesian observer model constrained by efficient coding can explain 'anti-Bayesian' percepts
Wei, X.-X., & Stocker, A. A. (2015) · Nature Neuroscience · doi:10.1038/nn.4105 · Cited by 397
Finding a roadmap to achieve large neuromorphic hardware systems
Hasler, J., & Marr, B. (2013) · Frontiers in Neuroscience · doi:10.3389/fnins.2013.00118 · Cited by 392
Neural networks: An overview of early research, current frameworks and new challenges
Prieto, A., Prieto, B., Ortigosa, E. M., Ros, E., Pelayo, F., Ortega, J., & Rojas, I. (2016) · Neurocomputing · doi:10.1016/j.neucom.2016.06.014 · Cited by 391
Deep Graph Contrastive Representation Learning
Zhu, Y., Xu, Y., Yu, F., Liu, Q., Wu, S., & Wang, L. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.04131 · Cited by 389
Deep Semi-Supervised Anomaly Detection
Ruff, L., Vandermeulen, R. A., Görnitz, N., Binder, A., Müller, E., Müller, K.-R., & Kloft, M. (2019) · arXiv · doi:10.48550/arxiv.1906.02694 · Cited by 388
Representation and separation of signals using nonlinear PCA type learning
Karhunen, J., & Joutsensalo, J. (1994) · Neural Networks · doi:10.1016/0893-6080(94)90060-4 · Cited by 371
Mutual Information, Fisher Information, and Population Coding
Brunel, N., & Nadal, J.-P. (1998) · Neural Computation · doi:10.1162/089976698300017115 · Cited by 350
Least mean square error reconstruction principle for self-organizing neural-nets
Xu, L. (1993) · Neural Networks · doi:10.1016/s0893-6080(05)80107-8 · Cited by 341
In search of common foundations for cortical computation
Phillips, W. A., & Singer, W. (1997) · Behavioral and Brain Sciences · doi:10.1017/s0140525x9700160x · Cited by 340
Evolutionary pursuit and its application to face recognition
Liu, C., & Wechsler, H. (2000) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.862196 · Cited by 323
Vision-Language Pre-Training with Triple Contrastive Learning
Yang, J., Duan, J., Tran, S., Xu, Y., Chanda, S., Chen, L., Zeng, B., Chilimbi, T., & Huang, J. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01522 · Cited by 309
Probabilistic framework for the adaptation and comparison of image codes
Lewicki, M. S., & Olshausen, B. A. (1999) · Journal of the Optical Society of America A · doi:10.1364/josaa.16.001587 · Cited by 308
Vision and the statistics of the visual environment
Simoncelli, E. P. (2003) · Current Opinion in Neurobiology · doi:10.1016/s0959-4388(03)00047-3 · Cited by 306
Gao, D., Han, S., & Vasconcelos, N. (2009) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2009.27 · Cited by 294
Mapping Information Flow in Sensorimotor Networks
Lungarella, M., & Sporns, O. (2006) · PLoS Computational Biology · doi:10.1371/journal.pcbi.0020144 · Cited by 293
Dynamic Model of Visual Recognition Predicts Neural Response Properties in the Visual Cortex
Rao, R. P. N., & Ballard, D. H. (1997) · Neural Computation · doi:10.1162/neco.1997.9.4.721 · Cited by 290
Information Flows in Causal Networks
AY, N., & POLANI, D. (2008) · Advances in Complex Systems · doi:10.1142/s0219525908001465 · Cited by 287
Unsupervised Attributed Multiplex Network Embedding
Park, C., Kim, D., Han, J., & Yu, H. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i04.5985 · Cited by 286
Nonlinear neurons in the low-noise limit: a factorial code maximizes information transfer
Nadal, J., & Parga, N. (1994) · Network Computation in Neural Systems · doi:10.1088/0954-898x/5/4/008 · Cited by 282
Unsupervised machine learning for exploratory data analysis in imaging mass spectrometry
Verbeeck, N., Caprioli, R. M., & Van de Plas, R. (2020) · Mass Spectrometry Reviews · doi:10.1002/mas.21602 · Cited by 281
Local Synaptic Learning Rules Suffice to Maximize Mutual Information in a Linear Network
Linsker, R. (1992) · Neural Computation · doi:10.1162/neco.1992.4.5.691 · Cited by 280
Self-citationAdaptive network for optimal linear feature extraction
Foldiak (1989) · International Joint Conference on Neural Networks · doi:10.1109/ijcnn.1989.118615 · Cited by 269
Simple codes versus efficient codes
Softky, W. R. (1995) · Current Opinion in Neurobiology · doi:10.1016/0959-4388(95)80032-8 · Cited by 267
Learning Factorial Codes by Predictability Minimization
Schmidhuber, J. (1992) · Neural Computation · doi:10.1162/neco.1992.4.6.863 · Cited by 264
Hyvärinen, A., & Hoyer, P. O. (2001) · Vision Research · doi:10.1016/s0042-6989(01)00114-6 · Cited by 261
Gradient descent learning algorithm overview: a general dynamical systems perspective
Baldi, P. (1995) · IEEE Transactions on Neural Networks · doi:10.1109/72.363438 · Cited by 258
How to Generate Ordered Maps by Maximizing the Mutual Information between Input and Output Signals
Linsker, R. (1989) · Neural Computation · doi:10.1162/neco.1989.1.3.402 · Cited by 255
Self-citationOn Mutual Information Maximization for Representation Learning
Tschannen, M., Djolonga, J., Rubenstein, P. K., Gelly, S., & Lučić, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.13625 · Cited by 253
Learning in linear neural networks: a survey
Baldi, P. F., & Hornik, K. (1995) · IEEE Transactions on Neural Networks · doi:10.1109/72.392248 · Cited by 252
Neural tuning and representational geometry
Kriegeskorte, N., & Wei, X.-X. (2021) · Nature Reviews Neuroscience · doi:10.1038/s41583-021-00502-3 · Cited by 187
Information Perspective to Probabilistic Modeling: Boltzmann Machines versus Born Machines
Cheng, S., Chen, J., & Wang, L. (2018) · Entropy · doi:10.3390/e20080583 · Cited by 104
Distributed Variational Representation Learning
Aguerri, I. E., & Zaidi, A. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2928806 · Cited by 84
Cognitive functions of intracellular mechanisms for contextual amplification
Phillips, W. A. (2017) · Brain and Cognition · doi:10.1016/j.bandc.2015.09.005 · Cited by 62
Machine learning action parameters in lattice quantum chromodynamics
Shanahan, P. E., Trewartha, D., & Detmold, W. (2018) · Physical Review D · doi:10.1103/physrevd.97.094506 · Cited by 65
An Information-Theoretic Perspective on Intrinsic Motivation in Reinforcement Learning: A Survey
Aubret, A., Matignon, L., & Hassas, S. (2023) · Entropy · doi:10.3390/e25020327 · Cited by 52
Contrastive Learning Inverts the Data Generating Process
Zimmermann, R. S., Sharma, Y., Schneider, S., Bethge, M., & Brendel, W. (2021) · arXiv · doi:10.48550/arxiv.2102.08850 · Cited by 52
Remembering forward: Neural correlates of memory and prediction in human motor adaptation
Scheidt, R. A., Zimbelman, J. L., Salowitz, N. M. G., Suminski, A. J., Mosier, K. M., Houk, J., & Simo, L. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2011.07.072 · Cited by 42
Guo, W., Huang, H., Kong, X., & He, R. (2019) · Proceedings of the 27th ACM International Conference on Multimedia · doi:10.1145/3343031.3351053 · Cited by 36
Hou, M., Xu, C., Liu, Y., Liu, W., Bian, J., Wu, L., Li, Z., Chen, E., & Liu, T.-Y. (2021) · Proceedings of the 30th ACM International Conference on Information & Knowledge Management · doi:10.1145/3459637.3482483 · Cited by 35
Cueva, C. J., Wang, P. Y., Chin, M., & Wei, X.-X. (2019) · arXiv · doi:10.48550/arxiv.1912.10189 · Cited by 27
Ford, N. (1989) · Journal of Information Science · doi:10.1177/016555158901500412 · Cited by 20
Metadata
- Authors
- Linsker, R.
- Venue
- Computer
- Published
- 1988-03
- Volume
- 21
- Issue
- 3
- Pages
- 105-117
- ISSN
- 0018-9162 (ISSN-L)
- DOI
- 10.1109/2.36
- OpenAlex
- W2122925692
- Cited by
- 1,469
- References
- 15
- FWCI
- 51.35
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.