Independent Component Analysis Using an Extended Infomax Algorithm for Mixed Subgaussian and Supergaussian Sources
Abstract
An extension of the infomax algorithm of Bell and Sejnowski (1995) is presented that is able blindly to separate mixed signals with sub- and supergaussian source distributions. This was achieved by using a simple type of learning rule first derived by Girolami (1997) by choosing negentropy as a projection pursuit index. Parameterized probability distributions that have sub- and supergaussian regimes were used to derive a general learning rule that preserves the simple architecture proposed by Bell and Sejnowski (1995), is optimized using the natural gradient by Amari (1998), and uses the stability analysis of Cardoso and Laheld (1996) to switch between sub- and supergaussian regimes. We demonstrate that the extended infomax algorithm is able to separate 20 sources with a variety of source distributions easily. Applied to high-dimensional data from electroencephalographic recordings, it is effective at separating artifacts such as eye blinks and line noise from weaker electrical signals that arise from sources in the brain.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Extended infomax independent components analysis (ICA) was applied to detect eye movement-related artifacts.
“Subsequently, extended infomax independent components analysis (ICA)63,64 was applied to detect eye movement-related artifacts.”
From Event-related potentials of automatic imitation are modulated by ethnicity during stimulus processing, but not during motor execution (2018) · cites this paper as [64]
Cites
Elements of Information Theory
Cover, T. M., & Thomas, J. A. (2001) · Wiley eBooks · doi:10.1002/0471200611 · Cited by 37,139 · see citing works
An Information-Maximization Approach to Blind Separation and Blind Deconvolution
Bell, A. J., & Sejnowski, T. J. (1995) · Neural Computation · doi:10.1162/neco.1995.7.6.1129 · Cited by 8,986 · see citing works
Independent component analysis: algorithms and applications
Hyvärinen, A., & Erkki Oja, A. (2000) · Neural Networks · doi:10.1016/s0893-6080(00)00026-5 · Cited by 8,726 · see citing works
Independent component analysis, A new concept?
Comon, P. (1994) · Signal Processing · doi:10.1016/0165-1684(94)90029-9 · Cited by 8,150 · see citing works
A Fast Fixed-Point Algorithm for Independent Component Analysis
Hyvärinen, A., & Erkki Oja, A. (1997) · Neural Computation · doi:10.1162/neco.1997.9.7.1483 · Cited by 3,373 · see citing works
Blind beamforming for non-gaussian signals
Cardoso, J.-F., & Souloumiac, A. (1993) · IEE Proceedings F Radar and Signal Processing · doi:10.1049/ip-f-2.1993.0054 · Cited by 2,759 · see citing works
Natural Gradient Works Efficiently in Learning
Амари, Ш.-И. (1998) · Neural Computation · doi:10.1162/089976698300017746 · Cited by 2,745 · see citing works
Blind separation of sources, part I: An adaptive algorithm based on neuromimetic architecture
Jutten, C., & Hérault, J. (1991) · Signal Processing · doi:10.1016/0165-1684(91)90079-x · Cited by 2,550 · see citing works
Independent Component Analysis of Electroencephalographic Data
Makeig, S., Bell, A. J., Jung, T., & B, T. J. S. A. (1995) · Cited by 1,957 · see citing works
A New Learning Algorithm for Blind Signal Separation
Амари, Ш.-И., Cichocki, A. S., & Yang, H. H. (1995) · Cited by 1,904 · see citing works
Blind signal separation: statistical principles
Cardoso, J.-F. (1998) · Proceedings of the IEEE · doi:10.1109/5.720250 · Cited by 1,886 · see citing works
III. Contributions to the mathematical theory of evolution
Pearson, K. (1894) · Philosophical Transactions of the Royal Society of London (A ) · doi:10.1098/rsta.1894.0003 · Cited by 1,571 · see citing works
Equivariant adaptive source separation
Cardoso, J.-F., & Laheld, B. H. (1996) · IEEE Transactions on Signal Processing · doi:10.1109/78.553476 · Cited by 1,357 · see citing works
Jung, T., Makeig, S., Westerfield, M., Townsend, J., Courchesne, E., & B, T. J. S. A. (2000) · Clinical Neurophysiology · doi:10.1016/s1388-2457(00)00386-2 · Cited by 1,326 · see citing works
Blind separation of auditory event-related brain responses into independent components
Makeig, S., Jung, T., Bell, A. J., Ghahremani, D. G., & B, T. J. S. A. (1997) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.94.20.10979 · Cited by 1,211 · see citing works
Independent Component Analysis: Theory and Applications
Lee, T.-W. (2013) · Cited by 711 · see citing works
Self-citationInfomax and maximum likelihood for blind source separation
Cardoso, J.-F. (1997) · IEEE Signal Processing Letters · doi:10.1109/97.566704 · Cited by 681 · see citing works
Kendall's Advanced Theory of Statistics, Volume 1: Distribution Theory.
Zacks, S., Stuart, A., & Ord, J. K. (1989) · Journal of the American Statistical Association · doi:10.2307/2289685 · Cited by 519 · see citing works
Blind separation of mixture of independent sources through a quasi-maximum likelihood approach
Pham, D. T., & Garat, P. (1997) · IEEE Transactions on Signal Processing · doi:10.1109/78.599941 · Cited by 472 · see citing works
Spatially independent activity patterns in functional MRI data during the Stroop color-naming task
McKeown, M. J., Jung, T., Makeig, S., Brown, G., Kindermann, S. S., Lee, T.-W., & B, T. J. S. A. (1998) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.95.3.803 · Cited by 461 · see citing works
Self-citationExtended ICA Removes Artifacts from Electroencephalographic Recordings
Jung, T., Humphries, C., Lee, T.-W., Makeig, S., McKeown, M. J., Iragui, V. J., & B, T. J. S. A. (1997) · Cited by 425 · see citing works
Self-citationBlind source separation of more sources than mixtures using overcomplete representations
Lee, T.-W., Lewicki, M. S., Girolami, M., & B, T. J. S. A. (1999) · IEEE Signal Processing Letters · doi:10.1109/97.752062 · Cited by 405 · see citing works
Self-citationA class of neural networks for independent component analysis
Karhunen, J., Erkki Oja, A., Wang, L., Vigário, R., & Joutsensalo, J. (1997) · IEEE Transactions on Neural Networks · doi:10.1109/72.572090 · Cited by 402 · see citing works
A unifying information-theoretic framework for independent component analysis
Lee, T.-W., Girolami, M., Bell, A. J., & B, T. J. S. A. (2000) · Computers & Mathematics with Applications · doi:10.1016/s0898-1221(00)00101-2 · Cited by 351 · see citing works
Self-citationAn Information-Theoretic Approach to Neural Computing
Deco, G., & Obradović, D. (1996) · Perspectives in neural computing · doi:10.1007/978-1-4612-4016-7 · Cited by 296 · see citing works
Stability Analysis of Learning Algorithms for Blind Source Separation
Амари, Ш.-И., Chen, T., & Cichocki, A. S. (1997) · Neural Networks · doi:10.1016/s0893-6080(97)00039-7 · Cited by 293 · see citing works
Nonlinear neurons in the low-noise limit: a factorial code maximizes information transfer
Nadal, J., & Parga, N. (1994) · Network Computation in Neural Systems · doi:10.1088/0954-898x_5_4_008 · Cited by 254 · see citing works
Blind source separation-semiparametric statistical approach
Амари, Ш.-И., & Cardoso, J.-F. (1997) · IEEE Transactions on Signal Processing · doi:10.1109/78.650095 · Cited by 244 · see citing works
The nonlinear PCA learning rule in independent component analysis
Oja, E. (1997) · Neurocomputing · doi:10.1016/s0925-2312(97)00045-3 · Cited by 220 · see citing works
Lee, T.-W., Lewicki, M. S., & B, T. J. S. A. (2000) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.879789 · Cited by 218 · see citing works
Self-citationDipole models of eye movements and blinks
Berg, P., & Scherg, M. (1991) · Electroencephalography and Clinical Neurophysiology · doi:10.1016/0013-4694(91)90154-v · Cited by 200 · see citing works
Robust learning algorithmfor blind separation of signals
Cichocki, A. S., Unbehauen, R., & Rummert, E. (1994) · Electronics Letters · doi:10.1049/el:19940956 · Cited by 194 · see citing works
Maximum likelihood for blind separation and deconvolution of noisy signals using mixture models
Moulines, É., Cardoso, J.-F., & Gassiat, É. (2002) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.1997.604649 · Cited by 179 · see citing works
McKeown, M. J. (2000) · NeuroImage · doi:10.1006/nimg.1999.0518 · Cited by 175 · see citing works
A Context-Sensitive Generalization of ICA
Pearlmutter, B. A., & Parra, L. C. (1996) · Maynooth University ePrints and eTheses Archive (Maynooth University) · Cited by 148 · see citing works
McKeown, M. J. (1998) · Medical Entomology and Zoology · Cited by 147 · see citing works
Chromatic structure of natural scenes
Wächtler, T., Lee, T.-W., & B, T. J. S. A. (2001) · Journal of the Optical Society of America A · doi:10.1364/josaa.18.000065 · Cited by 132 · see citing works
Self-citationNonlinear neurons in the low-noise limit: a factorial code maximizes information transfer
Nadal, J., & Parga, N. (1994) · Network Computation in Neural Systems · doi:10.1088/0954-898x/5/4/008 · Cited by 128 · see citing works
An Alternative Perspective on Adaptive Independent Component Analysis Algorithms
Girolami, M. (1998) · Neural Computation · doi:10.1162/089976698300016981 · Cited by 112 · see citing works
Self-citationNeural approaches to independent component analysis and source separation.
Karhunen, J. (1996) · Cited by 110 · see citing works
Blind source separation of nonlinear mixing models
Lee, T.-W., Köhler, B.-U., & Orglmeister, R. (2002) · Machine learning for signal processing ... · doi:10.1109/nnsp.1997.622422 · Cited by 94 · see citing works
Self-citationIndependent component analyses for quantifying neuronal ensemble interactions
Laubach, M., Shuler, M. G. H., & Nicolelis, M. A. L. (1999) · Journal of Neuroscience Methods · doi:10.1016/s0165-0270(99)00131-4 · Cited by 93 · see citing works
Independent Component Analysis: A Flexible Nonlinearity and Decorrelating Manifold Approach
Everson, R., & Roberts, S. (1999) · Neural Computation · doi:10.1162/089976699300016043 · Cited by 84 · see citing works
Multidimensional density shaping by sigmoids
Roth, Z., & Baram, Y. (1996) · IEEE Transactions on Neural Networks · doi:10.1109/72.536322 · Cited by 83 · see citing works
Faithful Representation of Separable Distributions
Lin, J. K., Grier, D. G., & Cowan, J. D. (1997) · Neural Computation · doi:10.1162/neco.1997.9.6.1305 · Cited by 82 · see citing works
Feature Extraction Through LOCOCODE
Hochreiter, S., & Schmidhuber, J. (1999) · Neural Computation · doi:10.1162/089976699300016629 · Cited by 82 · see citing works
Learning Nonlinear Overcomplete Representations for Efficient Coding
Lewicki, M. S., & Sejnowski, T. J. (1997) · Cited by 72 · see citing works
Self-citationColor-opponent receptive fields derived from independent component analysis of natural images
Tailor, D. R., Finkel, L. H., & Buchsbaum, G. (2000) · Vision Research · doi:10.1016/s0042-6989(00)00105-x · Cited by 66 · see citing works
Nonlinear source separation: the post-nonlinear mixtures.
Taleb, A., & Jutten, C. (1997) · Cited by 58 · see citing works
Girolami, M., & Fyfe, C. (1997) · IEE Proceedings - Vision Image and Signal Processing · doi:10.1049/ip-vis:19971418 · Cited by 57 · see citing works
Self-citationUnsupervised Classification with Non-Gaussian Mixture Models Using ICA
Lee, T.-W., Lewicki, M. S., & Sejnowski, T. J. (1998) · Cited by 56 · see citing works
Self-citationA maximum likelihood approach to nonlinear blind source separation
Pajunen, P., & Karhunen, J. (1997) · Lecture notes in computer science · doi:10.1007/bfb0020210 · Cited by 51 · see citing works
Nadal, J., & Parga, N. (1997) · Neural Computation · doi:10.1162/neco.1997.9.7.1421 · Cited by 43 · see citing works
Girolami, M., & Fyfe, C. (2002) · doi:10.1109/icnn.1997.614168 · Cited by 36 · see citing works
Self-citationIndependent component analysisapplied to feature extraction forrobust automatic speechrecognition
Potamitis, I., Fakotakis, N., & Kokkinakis, G. (2000) · Electronics Letters · doi:10.1049/el:20001365 · Cited by 31 · see citing works
Information backpropagation for blind separation of sources in nonlinear mixture
Yang, H. H., Амари, Ш.-И., & Cichocki, A. S. (2002) · doi:10.1109/icnn.1997.614237 · Cited by 26 · see citing works
von Hoff, T. P., Lindgren, A. G., & Kaelin, A. (2000) · Signal Processing · doi:10.1016/s0165-1684(00)00090-6 · Cited by 21 · see citing works
Non-linear independent component analysis using series reversion and Weierstrass network
Gao, P., Woo, W. L., & Dlay, S. S. (2006) · IEE Proceedings - Vision Image and Signal Processing · doi:10.1049/ip-vis:20045174 · Cited by 13 · see citing works
Weierstrass approach to blind source separation of multiple nonlinearly mixed signals
Gao, P., Woo, W. L., & Dlay, S. S. (2006) · IEE Proceedings - Circuits Devices and Systems · doi:10.1049/ip-cds:20050252 · Cited by 7 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Delorme, A., & Makeig, S. (2004) · Journal of Neuroscience Methods · doi:10.1016/j.jneumeth.2003.10.009 · Cited by 24,371 · see citing works
Brain Computer Interfaces, a Review
Nicolas-Alonso, L. F., & Gomez-Gil, J. (2012) · Sensors · doi:10.3390/s120201211 · Cited by 2,143 · see citing works
Independent EEG Sources Are Dipolar
Delorme, A., Palmer, J., Onton, J., Oostenveld, R., & Makeig, S. (2012) · PLoS ONE · doi:10.1371/journal.pone.0030135 · Cited by 810 · see citing works
The emergence of the social brain network: evidence from typical and atypical development
JOHNSON, M. H., GRIFFIN, R., CSIBRA, G., HALIT, H., FARRONI, T., DE HAAN, M., TUCKER, L. A., BARON–COHEN, S., & RICHARDS, J. (2005) · Development and Psychopathology · doi:10.1017/s0954579405050297 · Cited by 369 · see citing works
Learning-related changes in reward expectancy are reflected in the feedback-related negativity
Bellebaum, C., & Daum, I. (2008) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2008.06138.x · Cited by 299 · see citing works
Wessel, J. R., & Aron, A. R. (2015) · Psychophysiology · doi:10.1111/psyp.12374 · Cited by 244 · see citing works
The development of theta and alpha neural oscillations from ages 3 to 24 years
Cellier, D., Riddle, J., Petersen, I., & Hwang, K. (2021) · Developmental Cognitive Neuroscience · doi:10.1016/j.dcn.2021.100969 · Cited by 223 · see citing works
Gramann, K., Onton, J., Riccobon, D., Mueller, H. J., Bardins, S., & Makeig, S. (2010) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.2009.21369 · Cited by 172 · see citing works
Lenartowicz, A., Delorme, A., Walshaw, P. D., Cho, A. L., Bilder, R. M., McGough, J. J., McCracken, J. T., Makeig, S., & Loo, S. K. (2014) · The Journal of Neuroscience · doi:10.1523/jneurosci.1765-13.2014 · Cited by 158 · see citing works
Temporal dynamics of access to consciousness in the attentional blink
Kranczioch, C., Debener, S., Maye, A., & Engel, A. K. (2007) · NeuroImage · doi:10.1016/j.neuroimage.2007.05.044 · Cited by 143 · see citing works
Distinct neural generators of sensory gating in schizophrenia
Williams, T. J., Nuechterlein, K. H., Subotnik, K. L., & Yee, C. M. (2011) · Psychophysiology · doi:10.1111/j.1469-8986.2010.01119.x · Cited by 67 · see citing works
Baird, B., Castelnovo, A., Riedner, B. A., Lutz, A., Ferrarelli, F., Boly, M., Davidson, R. J., & Tononi, G. (2018) · eneuro · doi:10.1523/eneuro.0293-18.2018 · Cited by 45 · see citing works
Lucid dreaming occurs in activated rapid eye movement sleep, not a mixture of sleep and wakefulness
Baird, B., Tononi, G., & LaBerge, S. (2022) · Sleep · doi:10.1093/sleep/zsab294 · Cited by 41 · see citing works
Pfabigan, D. M., Alexopoulos, J., Bauer, H., Lamm, C., & Sailer, U. (2011) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2011.00100 · Cited by 41 · see citing works
Single trial prestimulus oscillations predict perception of the sound-induced flash illusion
Kaiser, M., Senkowski, D., Busch, N. A., Balz, J., & Keil, J. (2019) · Scientific Reports · doi:10.1038/s41598-019-42380-x · Cited by 31 · see citing works
The effect of alpha neurofeedback training on cognitive performance in healthy adults
Nawaz, R., Nisar, H., Yap, V. V., & Tsai, C.-Y. (2022) · Mathematics · doi:10.3390/math10071095 · Cited by 25 · see citing works
Vaughn, D. A., Maggiora, M. B., Vaughn, K. J., Maggiora, C. J., Tavakoli, A.-V., Liang, W., Zava, D., Cohen, M. S., & Lenartowicz, A. (2021) · Brain Research · doi:10.1016/j.brainres.2020.147203 · Cited by 13 · see citing works
Rauchbauer, B., Pfabigan, D. M., & Lamm, C. (2018) · Scientific Reports · doi:10.1038/s41598-018-30926-4 · Cited by 12 · see citing works
Močilnik, V., Rutar Gorišek, V., Sajovic, J., Pretnar Oblak, J., Drevenšek, G., & Rogelj, P. (2024) · Sensors · doi:10.3390/s24020329 · Cited by 8 · see citing works
Poghosyan, V., Ioannou, S., Al-Amri, K. M., Al-Mashhadi, S. A., Al-Mohammed, F., Al-Otaibi, T., & Al-Saeed, W. (2023) · Frontiers in Neuroscience · doi:10.3389/fnins.2023.948063 · Cited by 8 · see citing works
Rauchbauer, B., Lorenz, C., Lamm, C., & Pfabigan, D. M. (2021) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/s13415-021-00878-w · Cited by 7 · see citing works
- Authors
- Lee, T.-W., Girolami, M., & Sejnowski, T. J.
- Affiliations
- Salk Institute for Biological Studies (US)
- University of the West of Scotland (GB)
- Howard Hughes Medical Institute, Computational Neurobiology Laboratory, Salk Institute, La Jolla, CA 92037, U.S.A.
- ORCID
- Girolami, Mark
- Venue
- Neural Computation
- Publisher
- MIT Press
- Published
- 1999-02-01
- Volume
- 11
- Issue
- 2
- Pages
- 417-441
- ISSN
- 0899-7667 (ISSN-L)
- 1530-888X
- OpenAlex
- W2105909330
- Cited by
- 1,967
- References
- 59
- Citations by year
- 2026: 84 · 2025: 78 · 2024: 90 · 2023: 58 · 2022: 53 · 2021: 93 · 2020: 68 · 2019: 85 · 2018: 94 · 2017: 84 · 2016: 73 · 2015: 85 · 2014: 87 · 2013: 84 · 2012: 76
- FWCI
- 60.35
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Neural Computation (publishedVersion)
- PubMed (publishedVersion)
- UCL Discovery (University College London) (submittedVersion)
- Cambridge University Engineering Department Publications Database (submittedVersion)
- Funders
- Howard Hughes Medical Institute
- Topics
- Physical Sciences › Computer Science › Signal Processing › Blind Source Separation Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Life Sciences › Neuroscience › Cognitive Neuroscience › EEG and Brain-Computer Interfaces
- Keywords
- independent component analysis, Infomax algorithm, negentropy, projection pursuit, natural gradient, learning rules, source separation, EEG artifact removal, eye blink artifacts, line noise, high-dimensional data
- MeSH terms
- Algorithms, Blinking, Brain, Electroencephalography, Humans, Models, Statistical, Normal Distribution, Artifacts
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
