The DOI matches its Crossref record; 53 of the 53 references it cites were found in a registry or an index.
An Information-Maximization Approach to Blind Separation and Blind Deconvolution
Abstract
We derive a new self-organizing learning algorithm that maximizes the information transferred in a network of nonlinear units. The algorithm does not assume any knowledge of the input distributions, and is defined here for the zero-noise limit. Under these conditions, information maximization has extra properties not found in the linear case (Linsker 1989). The nonlinearities in the transfer function are able to pick up higher-order moments of the input distributions and perform something akin to true redundancy reduction between units in the output representation. This enables the network to separate statistically independent components in the inputs: a higher-order generalization of principal components analysis. We apply the network to the source separation (or cocktail party) problem, successfully separating unknown mixtures of up to 10 speakers. We also show that a variant on the network architecture is able to perform blind deconvolution (cancellation of unknown echoes and reverberation in a speech signal). Finally, we derive dependencies of information transfer on time delays. We suggest that information maximization provides a unifying framework for problems in "blind" signal processing.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Subsequently, extended infomax independent components analysis (ICA)63,64 was applied to detect eye movement-related artifacts.”
From Event-related potentials of automatic imitation are modulated by ethnicity during stimulus processing, but not during motor execution (2018) · cites this paper as [63]
In the citing paper
“Independent component analysis (ICA) decomposes the observed response into a linear mixture of non-Gaussian causes [36].”
From Hierarchical Models in the Brain (2008) · cites this paper as [36]
In the citing paper
“Next, we ran the Infomax ICA algorithm [Bell and Sejnowski, 1995], repeating it 15 times in ICASSO, implemented in GIFT, to generate a stable set of 75 final components.”
From Specific default mode subnetworks support mentalizing as revealed through opposing network recruitment by social and semantic FMRI tasks: Mentalizing and Default-Mode Subnetworks (2015) · cites this paper as Bell and Sejnowski, 1995
Citations
Cites
Neural Networks: A Comprehensive Foundation
Haykin, S. (1998) · Cited by 30,070
Probability, Random Variables, and Stochastic Processes.
Abrahams, J., & Papoulis, A. (1984) · Journal of the American Statistical Association · doi:10.2307/2288754 · Cited by 16,301
Independent component analysis, A new concept?
Comon, P. (1994) · Signal Processing · doi:10.1016/0165-1684(94)90029-9 · Cited by 8,324
A Fast Fixed-Point Algorithm for Independent Component Analysis
Hyvärinen, A., & Erkki Oja, A. (1997) · Neural Computation · doi:10.1162/neco.1997.9.7.1483 · Cited by 3,373
Blind separation of sources, part I: An adaptive algorithm based on neuromimetic architecture
Jutten, C., & Hérault, J. (1991) · Signal Processing · doi:10.1016/0165-1684(91)90079-x · Cited by 2,550
Adaptive filter theory (2nd ed.)
Haykin, S. (1991) · Prentice-Hall, Inc eBooks · Cited by 1,905
Barlow, H. B. (1989) · The MIT Press eBooks · doi:10.1162/neco.1989.1.3.295 · Cited by 1,709
First- and Second-Order Methods for Learning: Between Steepest Descent and Newton's Method
Battiti, R. (1992) · Neural Computation · doi:10.1162/neco.1992.4.2.141 · Cited by 1,193
What Is the Goal of Sensory Coding?
Field, D. J. (1994) · Neural Computation · doi:10.1162/neco.1994.6.4.559 · Cited by 1,183
A Simple Coding Procedure Enhances a Neuron's Information Capacity
Laughlin, S. B. (1981) · Zeitschrift für Naturforschung C · doi:10.1515/znc-1981-9-1040 · Cited by 1,015
Separation of a mixture of independent signals using time delayed correlations
Molgedey, L., & Schuster, H. G. (1994) · Physical Review Letters · doi:10.1103/physrevlett.72.3634 · Cited by 882
Could information theory provide an ecological theory of sensory processing?
Atick, J. J. (2011) · Network: Computation in Neural Systems · doi:10.3109/0954898x.2011.638888 · Cited by 679
Self-organizing neural network that discovers surfaces in random-dot stereograms
Becker, S., & Hinton, G. E. (1992) · Nature · doi:10.1038/355161a0 · Cited by 505
Space or time adaptive signal processing by neural network models
Hérault, J., & Jutten, C. (1986) · AIP conference proceedings · doi:10.1063/1.36258 · Cited by 406
A class of neural networks for independent component analysis
Karhunen, J., Erkki Oja, A., Wang, L., Vigário, R., & Joutsensalo, J. (1997) · IEEE Transactions on Neural Networks · doi:10.1109/72.572090 · Cited by 402
Blind separation of sources, part II: Problems statement
Comon, P., Jutten, C., & Hérault, J. (1991) · Signal Processing · doi:10.1016/0165-1684(91)90080-3 · Cited by 401
Representation and separation of signals using nonlinear PCA type learning
Karhunen, J., & Joutsensalo, J. (1994) · Neural Networks · doi:10.1016/0893-6080(94)90060-4 · Cited by 374
Local Synaptic Learning Rules Suffice to Maximize Mutual Information in a Linear Network
Linsker, R. (1992) · Neural Computation · doi:10.1162/neco.1992.4.5.691 · Cited by 280
Learning Factorial Codes by Predictability Minimization
Schmidhuber, J. (1992) · Neural Computation · doi:10.1162/neco.1992.4.6.863 · Cited by 279
Adaptation and decorrelation in the cortex
Barlow, H. B., & Földiák, P. (1989) · Addison-Wesley Longman Publishing Co., Inc. eBooks · Cited by 263
Cichocki, A. S., & Unbehauen, R. (1996) · IEEE Transactions on Circuits and Systems I Fundamental Theory and Applications · doi:10.1109/81.542280 · Cited by 259
Biswal, B. B., & Ulmer, J. L. (1999) · Journal of Computer Assisted Tomography · doi:10.1097/00004728-199903000-00016 · Cited by 255
Nonlinear neurons in the low-noise limit: a factorial code maximizes information transfer
Nadal, J., & Parga, N. (1994) · Network Computation in Neural Systems · doi:10.1088/0954-898x_5_4_008 · Cited by 254
Could information theory provide an ecological theory of sensory processing?
Atick, J. J. (1992) · Network Computation in Neural Systems · doi:10.1088/0954-898x/3/2/009 · Cited by 234
Blind separation of sources: A nonlinear neural algorithm
Burel, G. (1992) · Neural Networks · doi:10.1016/s0893-6080(05)80090-5 · Cited by 216
Olfactory computation and object perception.
Hopfield, J. J. (1991) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.88.15.6462 · Cited by 190
Criteria for multichannel signal separation
Yellin, D., & Weinstein, E. (1994) · IEEE Transactions on Signal Processing · doi:10.1109/78.301850 · Cited by 185
Neural networks expand SP's horizons
Haykin, S. (1996) · IEEE Signal Processing Magazine · doi:10.1109/79.487040 · Cited by 161
An Application of the Principle of Maximum Information Preservation to Linear Systems
Linsker, R. (1988) · Cited by 150
Statistical Independence and Novelty Detection with Information Preserving Nonlinear Maps
Parra, L. C., Deco, G., & Miesbach, S. (1996) · Neural Computation · doi:10.1162/neco.1996.8.2.260 · Cited by 141
Blind separation of sources, part III: Stability analysis
Sorouchyari, E. (1991) · Signal Processing · doi:10.1016/0165-1684(91)90081-s · Cited by 138
Nonlinear neurons in the low-noise limit: a factorial code maximizes information transfer
Nadal, J., & Parga, N. (1994) · Network Computation in Neural Systems · doi:10.1088/0954-898x/5/4/008 · Cited by 128
Nonlinear higher-order statistical decorrelation by volume-conserving neural architectures
Deco, G., & Brauer, W. (1995) · Neural Networks · doi:10.1016/0893-6080(94)00108-x · Cited by 122
Multidimensional density shaping by sigmoids
Roth, Z., & Baram, Y. (1996) · IEEE Transactions on Neural Networks · doi:10.1109/72.536322 · Cited by 83
Convergent Algorithm for Sensory Receptive Field Development
Atick, J. J., & Redlich, A. (1993) · Neural Computation · doi:10.1162/neco.1993.5.1.45 · Cited by 80
Current-mode subthreshold MOS implementation of the Herault-Jutten autoadaptive network
Cohen, M. H., & Andreou, A. G. (1992) · IEEE Journal of Solid-State Circuits · doi:10.1109/4.133158 · Cited by 59
CMOS Integration of Herault-Jutten Cells for Separation of Sources
Vittoz, E. A., & Arreguit, X. (1989) · Kluwer international series in engineering and computer science · doi:10.1007/978-1-4613-1639-8_3 · Cited by 48
Networks for the Separation of Sources that are Superimposed and Delayed
Platt, J. C., & Faggin, F. (1991) · Cited by 41
Redundancy reduction with information-preserving nonlinear maps
Parra, L. C., Deco, G., & Miesbach, S. (1995) · Network Computation in Neural Systems · doi:10.1088/0954-898x_6_1_004 · Cited by 38
Li, S., & Sejnowski, T. J. (1995) · IEEE Journal of Oceanic Engineering · doi:10.1109/48.380244 · Cited by 34
Self-citationAnalog CMOS integration and experimentation with an autoadaptive independent component analyzer
Cohen, M. H., & Andreou, A. G. (1995) · IEEE Transactions on Circuits and Systems II Analog and Digital Signal Processing · doi:10.1109/82.365346 · Cited by 33
A Non-linear Information Maximisation Algorithm that Performs Blind Separation
Bell, A. J., & Sejnowski, T. J. (1994) · Cited by 31
Self-citationA temporal model of linear anti-Hebbian learning
Girolami, M., & Fyfe, C. (1996) · Neural Processing Letters · doi:10.1007/bf00426022 · Cited by 28
Optimal Sampling of Natural Images: A Design Principle for the Visual System
Bialek, W., Ruderman, D., & Zee, A. (1990) · neural information processing systems · Cited by 27
Competitive Anti-Hebbian Learning of Invariants
Schraudolph, N. N., & B, T. J. S. A. (1991) · Cited by 26
Adaptive approach to blind source separation with cancellation of additive and convolutional noise
Cichocki, A. S., Kasprzak, W., & Амари, Ш.-И. (2002) · International Conference on Signal Processing · doi:10.1109/icsigp.1996.567290 · Cited by 17
Redundancy reduction with information-preserving nonlinear maps
Parra, L. C., Deco, G., & Miesbach, S. (1995) · Network Computation in Neural Systems · doi:10.1088/0954-898x/6/1/004 · Cited by 12
Schuster, H. G. (1992) · Physical Review A · doi:10.1103/physreva.46.2131 · Cited by 11
Blind equalization formulated as a self-organized learning process
Haykin, S. (2003) · Asilomar Conference on Signals, Systems & Computers (ACSSC) · doi:10.1109/acssc.1992.269177 · Cited by 8
Girolami, M., & Fyfe, C. (1996) · Elsevier eBooks · doi:10.1016/b978-044482587-2/50031-8 · Cited by 7
Príncipe, J. C., Wang, C., & Wu, H. (2002) · doi:10.1109/nnsp.1996.548371 · Cited by 7
Fynmore, R. J. (1908) · Notes and Queries · doi:10.1093/nq/s10-ix.229.393g · Cited by 5
Multi-Dimensional Density Shaping by Sigmoidal Networks
Baram, Y., & Roth, Z. (2005) · doi:10.1109/icnn.1995.487270 · Cited by 2
Flagged first, then by citations
Research works printed but not linked: 1
- Li S., 1994, I E E E J. Oceanic Eng., 20, 73
Cited by
Sorted by citations
Delorme, A., & Makeig, S. (2004) · Journal of Neuroscience Methods · doi:10.1016/j.jneumeth.2003.10.009 · Cited by 24,371
Deep learning in neural networks: an overview
Schmidhuber, J. (2015) · Neural Networks · doi:10.1016/j.neunet.2014.09.003 · Cited by 18,570
Learning the parts of objects by non-negative matrix factorization
Lee, D. D., & Seung, H. S. (1999) · Nature · doi:10.1038/44565 · Cited by 14,137
A Global Geometric Framework for Nonlinear Dimensionality Reduction
Tenenbaum, J. B., Silva, V. D., & Langford, J. C. (2000) · Science · doi:10.1126/science.290.5500.2319 · Cited by 13,585
$rm K$-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation
Aharon, M., Elad, M., & Bruckstein, A. M. (2006) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2006.881199 · Cited by 9,464
Metadata
- Authors
- Bell, A. J., & Sejnowski, T. J.
- Affiliations
- Howard Hughes Medical Institute (US)
- Howard Hughes Medical Institute, Computational Neurobiology Laboratory, The Salk Institute, 10010 N. Torrey Pines Road, La Jolla, CA 92037 USA and Department of Biology, University of California, San Diego, La Jolla, CA 92093 USA
- ORCID
- Bell, Anthony J.
- Venue
- Neural Computation
- Publisher
- MIT Press
- Published
- 1995-11
- Volume
- 7
- Issue
- 6
- Pages
- 1129-1159
- ISSN
- 0899-7667 (ISSN-L)
- 1530-888X
- OpenAlex
- W2108384452
- Cited by
- 8,986
- References
- 52
- Citations by year
- 2026: 151 · 2025: 242 · 2024: 264 · 2023: 286 · 2022: 329 · 2021: 375 · 2020: 349 · 2019: 372 · 2018: 343 · 2017: 334 · 2016: 346 · 2015: 357 · 2014: 365 · 2013: 352 · 2012: 328
- FWCI
- 130.90
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Neural Computation (publishedVersion)
- PubMed (publishedVersion)
- Funders
- Office of Naval Research
- Topics
- Physical Sciences › Computer Science › Signal Processing › Blind Source Separation Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
- Keywords
- information maximization, blind source separation, blind deconvolution, self-organized learning, nonlinear neural networks, higher-order statistics, redundancy reduction, principal component analysis, cocktail party problem, time delay dependence, blind signal processing
- MeSH terms
- Algorithms, Humans, Learning, Neurons, Probability, Problem Solving, Speech, Models, Statistical, Neural Networks, Computer
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
