The DOI matches its OpenAlex record; 35 of the 35 references it cites were found in a registry or an index.
Nonlinear neurons in the low-noise limit: a factorial code maximizes information transfer
Abstract
We investigate the consequences of maximizing information transfer in a simple neural network (one input layer, one output layer), focusing on the case of nonlinear transfer functions. We assume that both receptive fields (synaptic efficacies) and transfer functions can be adapted to the environment. The main result is that, for bounded and invertible transfer functions, in the case of a vanishing additive output noise, and no input noise, maximization of information (Linsker's infomax principle) leads to a factorial code-hence to the same solution as required by the redundancy-reduction principle of Barlow. We also show that this result is valid for linear and, more generally, unbounded, transfer functions, provided optimization is performed under an additive constraint, i.e. which can be written as a sum of terms, each one being specific to one output neuron. Finally, we study the effect of a non-zero input noise. We find that, to first order in the input noise, assumed to be small in comparison with the (small) output noise, the above results are still valid, provided the output noise is uncorrelated from one neuron to the other.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A Mathematical Theory of Communication
Shannon, C. E. (1948) · Bell System Technical Journal · doi:10.1002/j.1538-7305.1948.tb01338.x · Cited by 88,486
A Mathematical Theory of Communication
Shannon, C. E. (1948) · Bell System Technical Journal · doi:10.1002/j.1538-7305.1948.tb00917.x · Cited by 48,949
The Mathematical Theory of Communication
Shannon, C. E., Weaver, W., & Wiener, N. (1950) · Physics Today · doi:10.1063/1.3067010 · Cited by 9,801
Neural Information Processing Systems
(2018) · The Deep Learning Revolution · doi:10.7551/mitpress/11474.003.0014 · Cited by 3,730
Relations between the statistics of natural images and the response properties of cortical cells
Field, D. J. (1987) · Journal of the Optical Society of America A · doi:10.1364/josaa.4.002379 · Cited by 3,241
Some informational aspects of visual perception.
Attneave, F. (1954) · Psychological Review · doi:10.1037/h0054663 · Cited by 2,890
(1961) · Information and Control · doi:10.1016/s0019-9958(61)80064-8 · Cited by 2,875
Blind separation of sources, part I: An adaptive algorithm based on neuromimetic architecture
Jutten, C., & Herault, J. (1991) · Signal Processing · doi:10.1016/0165-1684(91)90079-x · Cited by 2,521
Russ, J. C. (2006) · CRC Press eBooks · doi:10.1201/9780203881095 · Cited by 2,107
Self-organization in a perceptual network
Linsker, R. (1988) · Computer · doi:10.1109/2.36 · Cited by 1,469
Cited by
Deep learning in neural networks: an overview
Schmidhuber, J. (2015) · Neural Networks · doi:10.1016/j.neunet.2014.09.003 · Cited by 18,570
An Information-Maximization Approach to Blind Separation and Blind Deconvolution
Bell, A. J., & Sejnowski, T. J. (1995) · Neural Computation · doi:10.1162/neco.1995.7.6.1129 · Cited by 8,986
Independent component analysis: algorithms and applications
Hyvärinen, A., & Oja, E. (2000) · Neural Networks · doi:10.1016/s0893-6080(00)00026-5 · Cited by 8,705
Removing electroencephalographic artifacts by blind source separation
Jung, T., Makeig, S., Humphries, C., Lee, T.-W., McKeown, M. J., Iragui, V. J., & Sejnowski, T. J. (2000) · Psychophysiology · doi:10.1017/s0048577200980259 · Cited by 3,334
Natural Gradient Works Efficiently in Learning
Amari, S.-I. (1998) · Neural Computation · doi:10.1162/089976698300017746 · Cited by 2,681
Metadata
- Authors
- Nadal, J., & Parga, N.
- Venue
- Network Computation in Neural Systems
- Publisher
- Taylor & Francis
- Published
- 1994-01-01
- Volume
- 5
- Issue
- 4
- Pages
- 565-581
- ISSN
- 0954-898X (ISSN-L)
- OpenAlex
- W2082195896
- Cited by
- 282
- References
- 35
- FWCI
- 10.19
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Nonlinear neurons in the low-noise limit: a factorial code maximizes information transfer. https://app.cochat.ai/papers/10.1088/0954-898x/5/4/008
- The paper
- Nadal, J., & Parga, N. (1994). Nonlinear neurons in the low-noise limit: a factorial code maximizes information transfer. Network Computation in Neural Systems. https://doi.org/10.1088/0954-898x/5/4/008
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.